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ABSTRACT 


This  report  discusses  the  research  conducted  by  the  research  team 
at  the  Georgia  Institute  of  Technology  in  the  area  of  nonserial  dynamic 
programming  networks.  The  problems,  approach  and  major  results  are 
summarized  in  Chapter  1,  while  the  rest  of  the  chapters  discuss,  in 
detail,  the  algorithm  developments,  experiments  with  sample  problems, 
and  algorithmic  complexities.  Each  chapter  contains  detailed  computer 
flow  charts  for  the  algorithms  developed.  Chapter  6  employs  an  efficient 
dimensionality  reduction  algorithm  known  as  the  imbedded  state  space 
method  in  conjunction  with  the  one  developed  in  this  study  to  treat  an 
otherwise  intractable  problem  involving  feedforward  loop  systems.  These 
algorithms  are  useful  inputs  to  the  development  of  dynamic  programming 
based  strategies  for  the  analysis  of  complex  nonserial  networks. 
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Chapter  1 


NONSERIAL  DYNAMIC  PROGRAMMING  NETWORKS:  THE  PROBLEM,  APPROACH,  AND  MAJOR  RESULTS 

1. 1  Introduction 

Our  overall  interest  is  in  the  optimal  analysis  (and/or  design)  of  large 
scale  systems.  Generally,  a  large  complex  system  is  composed  of  several  inter¬ 
connected  subsystems  which  individually  may  be  simpler  than  the  parent  system. 

In  nonserial  systems,  the  structure  of  these  interconnections  creates  further 
complexity.  Such  complexities,  for  example,  may  be  engendered  by  the  presence 
of  combinations  of  various  nonserial  networks. 

A  nonserial  system,  as  defined  by  Beightler  and  Meier  [1],  is  a  system  where 
at  least  one  subsystem  in  the  system  receives  inputs  from  more  than  one  sub¬ 
subsystem  or  sends  outputs  to  more  than  one  subsystem.  It  could  also  be  described 
as  a  system  where  for  at  least  one  of  the  stages,  the  output  is  not  the  input  to  the 
next;  thus,  there  exists  at  least  one  n  such  that  the  output  xr  ?  x^  the  input 
of  the  next  stage.  Nonserial  systems  are  encountered  in  the  study  of  chemical 
processing  systems,  natural  gas  transmission  pipelines,  water  resources  systems, 
energy,  production-inventory  systems,  and  various  other  systems.  Practical  examples 
of  these  systems  are  further  discussed  by  Esogbue  and  Marks  [14].  Thus,  there  exist 
important  reasons  to  study  such  systems. 

To  motivate  our  discussion,  let  us  consider  a  general  mathematical  formulation 
for  the  following  nonserial  system,  which  approximates  our  concept  of  a  large 
scale  system. 

1 . 2  A  General  Nonserial  System:  Some  Motivations 

For  illustrative  purposes,  we  introduce  the  following  general  complex  nonserial 
system  which  is  an  example  of  a  complex  network: 
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Figure  1.1:  An  Example  of  a  Large  Complex  Network 
In  the  foregoing,  let 

be  the  vector  of  inputs  co  subsystem  i,  and 
the  corresponding  vector  of  decision  variables,  e  1L. 

Further,  let  be  defined  as  the  set  of  output  variables  to  subsystem  i,  with 

z.  defined  as  z.  z  Z.,  and  f.(Y.,  M.)  as  the  objective  function  for  subsystem  i. 
i  11111  J 

In  this  representation,  we  may  visualize 

z.  -  $(Y.,  M.)  as  the  set  of  interconnection  relations  for  subsystem 
ill 

i  and  (Y^,  M  )  z  S^,  where  is  the  set  of  restrictions  for  subsystem  i.  Note  the 
generality  of  these  definitions  i.e.,  f  and  <S>  need  not  be  familiar 

functions. 

As  an  example,  consider  subsystem  3:  Y^  =  {x^,  *13^’  Z3  =  ^35’  x34^'  A 

mathematical  programming  formulation  for  this  nonserial  system  is  the  following: 

7 

max  E  f . (Y. ,  M, )  (1) 

i=l  1  1 

s.t.  z^  =  $(Y^,  IK)  V  z^  e  i  =  1,  2,  ....  7  (2) 

(Yi,  M  )  e  Si  i  =  1,  2,  . . . ,  7  .  (3) 

Note  that  this  complex  system  contains  subsets  of  various  classical  nonserial 
systems  such  as  diverging  branch,  converging  branch  and  feedback  loop  systems. 


and  as  such  may  be  considered  a  generalized  network.  Its  treatment  is  thus 
nontrival. 

When  the  objective  function  and  the  constraints  are  linear,  the  resulting 
mathematical  programming  problem  can  easily  be  solved  via  linear  programming 
methods.  When  the  objective  function  and  the  constraints  are  convex  functions 
defined  over  a  convex  set,  the  mathematical  programming  problem  becomes  a  convex 
programming  problem  for  which  there  are  methods  of  solution.  However,  when  any  of 
the  convexity  assumptions  are  dropped,  this  mathematical  programming  problem 
becomes  difficult  to  solve.  Dynamic  programming  does  not  depend  upon  the  nature  of 
the  objective  function,  constraints,  or  the  construct  of  the  domain  of  the  feasible 
region  of  search.  It  thus  possesses  some  appeal  for  solving  problems  of  this  genre. 

Dynamic  programming  has  been  used  to  optimize  nonserial  systems  in  various  areas. 
Wong  and  Larson  [27 ]  for  instance,  used  dynamic  programming  in  the  design  and  opera¬ 
tion  of  natural  gas  transmission  pipelines  with  a  diverging  branch  structure.  Mitten 
and  Nemhauser  [  19 J  applied  this  method  to  a  hypothetical  chemical  process  which  con¬ 
tained  a  recycle  feedback  loop  around  a  reactor.  Beightler  and  Meier  [ 2  ]  considered 
the  application  of  dynamic  programming  to  a  river  basin  reservoir  system  with  a 
converging  branch  structure.  Esogbue  and  Marks  [15]  studied  several  project 
scheduling  and  resource  allocation  problems  of  the  CPM-Cost  variety  in  which  the 
precedence  relationships  possess  a  nonserial  structure.  Some  efficient  dynamic 
programming  based  procedures  for  network  compression  were  advanced. 

Obviously,  more  real  life  systems  can  be  formulated  as  nonserial  dynamic 
programming  problems.  The  limited  invocation  of  the  method  for  the  analysis  of 
more  complex  structural  systems  is  attributable,  in  the  main,  to  the  lack  of  a  suffi¬ 
ciently  "n^"  order  theory  of  nonserial  systems  and  to  the  usual  computational 
problems  that  have  plagued  the  application  of  dynamic  programming.  Methods  for 
alleviating  the  computational  burden  so  frequently  encountered  can  be  found  in 
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another  paper  by  Esogbue  and  Marks  [14 ] . 
1.3  Previous  Related  Works 


A  number  of  contributors  to  the  state  of  knowledge  of  nonserial  dynamic 
programming  deserve  mention.  Wilde  and  Beightler  [26 ]  and  Nemhauser  [21  ]  reviewed 
the  basic  theory  involved  in  the  optimization  of  the  four  basic  classes  of  nonserial 
systems:  1)  diverging  branch  systems,  2)  converging  branch  systems,  3)  feedforward 
loop  systems,  and  4)  feedback  loop  systems.  More  complicated  nonserial  systems, 
however,  cannot  be  studied  unless  the  results  for  the  previously  mentioned  four 
systems  can  be  applied  to  the  problem.  The  example  in  Section  2,  for  instance, 
cannot  be  optimized  efficiently  using  the  method  of  nonserial  dynamic  programming 
contained  in  the  the  original  works  cited  above.  Simple  modifications  and  exten¬ 
sions  of  the  theory  are  not  helpful  either. 

One  reason  for  the  above  quandry  is  simply  that  this  system  is  not  composed  of 
a  simple  combination  of  the  above  mentioned  four  classes  of  nonserial  systems.  We 
are  thus  unable  to  determine  the  order  in  which  the  subsystems  should  be  optimized, 
so  that  the  dynamic  programming  procedure  can  be  performed  efficiently. 

Another  important  reason  is  that  computational  techniques  capable  of  being 
used  to  tackle  the  nonserial  examples  given  in  the  literature  do  not  exist.  Further¬ 
more,  the  issues  are  not  discussed  anywhere.  One  is  left  to  impute  that  traditional 
computational  problems  inhibiting  widespread  use  of  dynamic  programming  become 
exacerbated  in  the  nonserial  case. 

Among  the  major  contributors  to  the  current  literature  of  dyanmic  programming 
are  Bertele  and  Brioschi  [5,  6,  7,  8  ].  However,  their  main  concern  is  with  the 
optimization  of  a  problem  whose  objective  function  has  the  following  specialized  form 
min  F(X)  =  min  Z  f  (X1) 

X  X  ieT  (4) 
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in  which 

X  =  (x^ ,  X2 . xn>  is  a  set  of  discrete  variables,  a  being  the  (5) 

number  of  feasible  values  of  the  variable  x^. 

T  -  {1,  2,  ....  t}  and  X1  C  X.  (6) 

In  the  above,  the  function  F(X)  is  called  the  objective  function  and  the  functions 
f^X*)  are  the  components  of  the  objective  function.  Before  Bertele  and  Brioschi 
can  optimize  this  problem  via  dynamic  programming,  the  order  in  which  the  compo¬ 
nents  of  the  objective  function  are  to  be  optimized  by  the  dynamic  programming 
procedure  has  to  be  determined,  so  that  the  number  of  computations  for  the  problem 
can  be  minimized.  A  series  of  algorithms  using  graph  theory  concepts  to  determine 
this  optimal  order  is  then  developed.  While  these  contributions  are  important, 
the  limitations  of  the  problem  addressed  and  thus  the  algorithms  are  evident.  The 
other  contributors  to  the  computational  aspects  of  nonserial  dynamic  programming 
include  Beightler,  Johnson  and  Wilde  [  l),  Parker  [24],  Parker  and  Crisp  [25]  and 
Brown  [9].  The  superposition  approach  suggested  by  Beightler  et  al  [l  ]  for 
treating  converging  branch  systems  under  the  assumption  of  linear  return  and  transi¬ 
tion  functions  with  additive  compositor  operator  was  extended  to  nonlinear  con¬ 
verging  branch  systems  by  Parker  and  Crisp  [25]-  An  extension  of  some  of  these 
concepts  to  feedforward  and  feedback  loop  systems  was  presented  by  Parker.  In  [ 9  ] 
Brown  considers  a  different  approach  to  the  analysis  of  converging  branch  systems 
under  both  deterministic  and  stochastic  return  and  transition  functions.  The 
procedure  considers  the  nonserial  converging  branch  system  as  a  serial  system  by 
grouping  the  stages  of  the  two  branches  together  into  new  two  dimensional  stages. 

In  this  case, the  input  and  output  state  vectors,  as  well  as  the  decision  vectors, 
are  all  two  dimensional.  The  approach,  although  apparently  demanding  on  the  storage 
requirement,  has  interesting  features  especially  with  regards  to  the  analysis  of 
stochastic  systems. 
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Because  of  our  concern  for  complexity  reduction,  we  have  demonstrated  in  [15] 
that  an  adroit  combination  of  certain  potent  but  hitherto  isolated  concepts  and 
techniques  of  large  scale  problem  solving  can  lead  to  the  solution,  via  nonserial 
dynamic  programming,  of  certain  interesting  nonserial  systems.  Specifically, 
efficient  formulations  for  treating  a  three  branch  converging  system,  a  system 
with  multipaths  departing  from  a  junction,  and  a  complex  converging-diverging- 
converging  system,  were  developed  by  an  adroit  synthesis  of  the  pseudo  stage 
concept  (Beightler  and  Meier  [2]),  Nemhauser  and  Ullman's  method  [23]  and  an 
optimal  elimination  technique  akin  to  Bertele  and  Brioschi  [6  ].  This  method  proved 
to  be  considerably  more  efficient  than  the  solo  application  of  any  of  the  foregoing 
or  any  currently  available  algorithm. 

1.4  Research  Objectives 

The  overall  objective  of  this  research  is  to  extend  the  theory  of  nonserial 
dynamic  programming  so  that  it  can  be  applied  to  most  nonserial  systems.  In  [12],. 
we  demonstrated  that  the  following  three  factors  hinder  the  solution  of  a  problem 
by  dynamic  programming:  1)  the  amount  of  high  speed  memory  required  for  the 
problem  (space  complexity) ,  2)  the  total  number  of  calculations  required  (computa¬ 
tional  complexity),  and  3)  the  amount  of  off-line  memory  required  (space  complexity). 
With  these  criteria  in  mind  we  wish  to  develop  an  algorithm  which, in  addition,  will 
give  an  optimal  order  in  which  the  subsystems  in  any  nonserial  system  should  be 
optimized. 

This  is  based  on  the  premise  that  the  crucial  issue  in  nonserial  systems 
research  revolves  around  the  set  of  questions:  a)  given  a  complex  nonserial  system, 
how  do  we  collapse  it  into  a  serial-like  structure?  b)  in  the  branch  compression 
effort,  what  optimal  order  should  be  followed  in  order  for  the  resultant  dynamic 
program  to  be  efficient  with  regards  to  the  usual  issues  in  dynamic  programming 
algorithm  development?  While  Bertele  and  Brioschi  have  proposed  an  optimal  com¬ 
pression  order  for  a  highly  specialized  nonserial  form,  we  are  unaware  of  any 
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successful  attempts  to  generalize  or  develop  a  theory  applicable  to  most,  if  not 
all,  nonserial  forms.  While  we  would  like  to  move  in  this  direction,  we  thought 
that  our  initial  efforts  should  be  directed  to  the  development  of  efficient 
algorithms  for  the  treatment  of  the  classical  nonserial  systems  which  link  up, 
on  a  higher  level,  to  form  a  complex  nonserial  system.  These  algorithms  would 
be  accompanied  with  detailed  treatments  of  their  algorithmic  complexities.  Such 
analyses  were  particularly  missing  in  previous  works  in  the  literature.  They  are, 
however,  considered  absolutely  necessary  in  the  development  of  strategies  for 
resolving  complex  nonserial  network  problems. 

1.5  Research  Plan  and  Results 

The  research  consisted  of  the  following  six  phases: 

1.5.1)  Phase  1 

A  number  of  criteria  for  discussing  efficiency  exist.  These  are,  in  turn, 
contingent  on  the  choice  of  the  objective  function.  In  the  first  phase,  we  con¬ 
sidered  these  issues  and  determined  a  set  of  criteria  for  which  we  wished  to  develop 
an  algorithm.  They  were  based  upon  the  three  principal  factors  which  tend  to  hinder 
the  solution  of  a  problem  via  dynamic  programming  as  well  as  those  in  combinatorics. 
Several  criteria  were  examined  first  before  focusing  on  the  most  appropriate  set 
or  combinations.  Structural  characteristics  (attributes)  of  a  complex  nonserial 
system  of  the  type  depicted  in  Fig.  1. 1  were  studied  leading  to  a  grouping  of  complex 
nonserial  systems  by  attributes  and  by  degree  of  complexity.  For  example,  we 
developed  a  characterization  of  the  complexity  of  a  nonserial  network  in  terms  of 
the  following  parameters:  N,  the  number  of  nodes;  M,  the  connectedness,  and  ordering 
or  arc  orientation  such  as  diverging,  converging,  feedforward,  and  feedback  and 
various  combinations  of  each. 

Using  these  parameters,  the  complexities  may  be  described  as  follows: 
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1)  Large  N  and  simple  structure 

2)  Large  N  and  complex  combination  of  arc  orientations  such 


as  converging  branch,  diverging  branch,  feedforward  and 

feedback  loops. 

3)  Small  N  and  simple  structure 

A)  Small  N  and  complex  structure 

1.3.2)  Phase  2 

An  algorithm  was  then  developed  for  each  of  the  criteria  decided  upon  in 
Phase  1.  The  algorithms  were  structured  so  that  the  order  of  optimization  of  the 
subsystems  of  the  classical  nonserial  systems  was  the  best  possible.  We  invoked 
concepts  from  graph  theory,  signal  flow  graphs,  and  automata  theory  wherever  possible 
in  the  dynamic  programming  algorithm  development. 

Using  the  conventional  DP  algorithm,  a  computer  code  in  FORTRAN  V  was  developed 
for  both  the  diverging  and  converging  branch  systems.  Next,  detailed  computer 
algorithms  for  these  systems  were  constructed.  Using  algorithm  analysis,  results 
were  derived  to  describe  the  space  and  time  complexities  of  these  algorithms.  For 
example,  if  we  let  the  discretization  levels  for  the  state  and  decision  variables  be 
denoted  by  Kg  and  Kp  respectively,  and  further  let  M  be  the  number  of  stages  in  the 
subbranch  while  N  represents  the  number  in  the  main  serial  system,  then  we  obtain  the 
following  results: 

a)  For  the  Diverging  Branch  System 

The  maximum  space  requirement  »  (M+N+3)K_,  and 

2  2 

The  number  of  computations  =  (M  +N  +>H-N)  (Kn+1)K  +  4(lFfN)+3K  +  9 

b)  For  the  Converging  Branch  System 

2 

The  maximum  space  requirement  *  2K_  +  (MK  +  N+3)K  ,  and 

b  D  S 

2  2  2 

The  number  of  computations  -  [(K^+l)  (M  +N  +M+N)  +4)  Kg  +  (K^+4)Kg  + 

4M  +  4N  -  1 

l 

i 

i 
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These  derived  results  are  important  in  many  ways.  For  example,  they  may  be  used 
to  compare  the  computational  and  space  complexities  of  classical  nonserial  networks 
under  the  following  restrictions: 

i)  identical  number  of  branches 

ii)  identical  number  of  nodes  in  subbranch  as  well  as  in  the  main  serial 
system,  i.e.  N  =  2M 

iii)  discretization  levels  for  states  and  decisions  are  the  same, 
i.e.  Kp  -  Kg. 

The  following  comparisons  are  instructive: 

a)  Space  complexity  comparison: 

Space  for  diverging  branch  structure  =  3(M+l)Kg 

2 

Space  for  converging  branch  structure  =  (M+2)KC  +  (2M+3)K 

b)  Computational  complexity  comparison: 

Number  of  computations  for  diverging  branch  structure  = 

M(5M+3)Kg  +  (5M2+3M+3)Ks  +  8M  +  9 

Number  of  computations  for  converging  branch  structure  = 

M(5M+3)KC3  +  (5M+3M=5)K?  +  4KC  +  12M  -  1. 

b  ,  O  b 

We  sum  up  the  comparisons  as  follows:  (See  Table  1.1) 
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Table  1.1:  Comparison  of  Complexities  of  Diverging 
and  Converging  Branch  Systems 


Structure 

Complexity 

Name 

Complexity  as 
a  function  of 

Nature  of 
Complexity 

i 

Diverging  branch 

Space 

Number  of  nodes 
in  the  branch 

Linear 

Converging  branch 

Space 

Number  of  nodes 
in  the  branch 

Linear 

Diverging  branch 

Space 

Discretization  levels 

Linear 

Converging  branch 

Space 

Discretization  levels 

Quadratic 

Diverging  branch 

Computational 

Number  of  nodes 
in  the  branch 

Quadratic 

Converging  branch 

Computational 

Number  of  nodes 
in  the  branch 

Quadratic 

Diverging  branch 

Computational 

Discretization  levels 

Quadratic 

Converging  branch 

Computational 

Discretization  levels 

Cubic 

We  may  thus  conclude  that  both  from  space  and  computational  complexity  considera¬ 
tions,  the  diverging  branch  system  is  less  complex  than  the  converging  branch 
one  -  a  fact  that  is  suggested  by  the  problem  structure.  Detailed  analysis 
and  results  appear  in  the  respective  chapters. 

1.5.3)  Phase  3 

High  level  computer  programs  in  FORTRAN"  V  were  written  for  both  the  diverging 
branch  and  converging  branch  systems.  These  were  exemplified  via  several  test 
problems.  The  systems  were  subjected  to  perturbations  in  both  N  and  N  and  return 
as  well  as  transition  functions.  Their  algorithmic  complexities  were  experimentally 
verified  in  each  case.  Refinements  in  the  algorithms  were  executed  by  introducing 
concepts  such  as  branch  compression  and  node  elimination,  especially  for  complex 
multi-branch  systems.  Detailed  computer  flow  charts  for  each  problem  were  con¬ 
structed.  We  emphasize  the  utility  of  our  approach.  The  algorithms  admit  input. 
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return,  and  transition  data  in  terms  of  functions  of  the  associated  variables. 
Storage  problems  were  not  encountered  with  this  line  of  pursuit.  For  programs 
involving  data  structures  in  the  form  of  tables,  see  the  report  by  Dr.  N.  Warsi. 

1.5.4)  Phase  4 

In  this  phase,  the  exercises  of  phases  2  and  3  we-e  extended  to  the  more 
complex  systems  namely  the  feedforward  and  feedback  loop  systems. 

1.5.5)  Phase  5 

Because  of  our  interest  in  efficiency  of  algorithms,  we  explored  the  possi¬ 
bility  of  an  adroit  combination  of  algorithms  to  solve  problems  which  might  other¬ 
wise  defy  solution  via  a  solo  application  of  an  algorithm.  Thus,  we  constructed  a 
feedforward  loop  example  in  which  the  return  functions  and  the  constraints  possess 
a  special  structure.  In  particular,  the  problem  had  components  which  had  the  charac 
teristics  of  a  multidimensional  nonlinear  knapsack  problem.  This  problem,  as 
posed,  could  not  be  solved  without  major  modifications  in  our  algorithm.  It  was, 
however,  successfully  solved  using  a  combination  of  our  algorithm  and  the  imbedded 
state  space  dynamic  programming  routine. 

1.5.6  Phase  6 

Finally,  we  considered  the  following  assertion  which  we  developed  in  connection 
with  our  study  of  nonserial  networks  of  the  CPM-Cost  variety  using  the  project  cost 
minimization  criterion: 


Whenever  a  nettcoak  is  Audi  that  fan.  eac.fi  activity  i  and  j, 
with  activity  i  p.%eceding  activity  j  (i  <  j  }  the  Act  o 
pathi  p.  and  p  ■  containing  activities  i  and  j  xeApectivety 

is  given  by  p .  c  p.  on.  p  ■  c  p-  then  we  can  asAeat  that  the 

*L  j  J  ~  4. 

fanctional  equation  i$n(U  decompoACA  into  a  Aequcnce  OjJ  one 
dimensional  dynamic  pnogaamming  pfioblemA. 


This  is  a  sufficient  condition  which  we  can  prove  rather  easily.  We  attempted  to 
develop  a  condition  that  is  both  sufficient  and  necessary  with  the  hope  that  an 
algorithm  useful  in  the  analysis  of  more  complex  systems  may  result.  However, 


because  of  time  and  other  resource  constraints,  this  line  of  pursuit  was  truncated 
for  the  moment. 

1.6  Report  Outline 

Having  discussed  the  project  background,  purpose  and  major  results  in  this 
chapter,  we  dedicate  the  rest  of  the  report  to  the  development  of  the  dynamic  pro¬ 
gramming  analysis  of  the  various  classical  nonserial  systems.  We  begin  with  the 
diverging  branch  system,  the  easiest  of  the  systems  to  analyze.  This  is  treated  in 
Chapter  2.  The  algorithm,  flowcharts,  sample  problems  and  algorithmic  analyses 
are  given.  In  Chapter  3,  the  converging  branch  system  is  treated.  This  chapter 
includes  two  algorithms  -  the  original  and  a  modified  one  for  the  purpose  of 
solving  problems  with  complex  functions.  In  Chapter  4,  nonserial  systems  with 
feedforward  loop  structures  are  discussed.  The  loop  system  is  very  complex 
since  it  involves  a  combination  of  the  diverging  and  converging  branch  systems. 
Chapter  5  considers  the  feedback  loop  nonserial  problem.  This  is  akin  to  the 
feedforward  in  many  ways.  The  report  is  concluded  with  Chapter  6.  In  this 
chapter,  a  feedforward  loop  example  was  constructed  involving  return  and  transi¬ 
tion  functions  that  were  different  from  any  type  considered  previously.  In 
particular,  the  return  functions  were  step  functions  and  the  resource  availability 
constraints  were  expressed  in  terras  of  constants.  The  imbedded  state  space  algorithm 
was  used  in  conjunction  with  our  algorithm  to  solve  this  problem.  The  algorithmic 
steps  are  provided  but  computer  implementation  was  truncated  because  of  budgetary  and 
time  constraints  on  this  project.  It  could  be  seen  though  that  this  is  a  beneficial 
and  necessary  line  of  pursuit  when  structurally  complex  nonserial  systems  are 
being  investigated. 
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Chapter  2 

ANALYSIS  OF  DIVERGING  BRANCH  NONSERIAL  NETWORKS 


2.1  Development  of  a  Dynamic  Programming  Algorithm  for  the  Diverging  Branch  System 
A  diverging  branch  system  (see  Fig.  2.1)  is  the  easiest  of  the  elementary 
nonserial  structures  to  analyze.  For  simplicity,  we  first  consider  a  two  branch 
system.  The  stage  transformations  and  return  functions  both  for  a  main  serial 


process  and  for  a  branch  are  defined  as  follows: 


and 


x  .  =  t  (x  ,  d  ) 
n-1  n  n  n 

1 

n  = 

i. 

2,  ....  N 

Vl.l  -  Eml^xml*  d„l> 

> 

m  = 

i. 

2 ,  ....  M 

*M1  =  tSl^xS ’  ‘V 

r  ■  r  (x  ,  d  ) 
n  n  n  n 

» 

n  = 

i. 

2 . N 

r  .  =  r  (x  ,  d  ) 

m,l  m,l  ml  ml 

1 

m  = 

i. 

2 . M 

Figure  2.1:  A  Diverging  Branch  System 
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Consider  the  basic  system  consisting  of  one  main  serial  system 
“  It  2,  ••• i  N)  and  one  branch  (M  =  1,  2,  . ..,  M).  Let  us  assume  that  the 
input  and  decision  variables  at  each  stage  have  the  following  integer  values  : 


1  i  *ii  i  Ku 
1  i  du  ±  pu 


i  =  1,  2,  ....  M 
i  =  1,  2,  ... ,  N 

i  -  1,  2 . M 

1  B  1 i  2 ,  ...»  N 


To  develop  our  algorithm,  we  first  decompose  the  network  into  four  phases 
and  then  we  employ  the  usual  recursive  procedures  in  optimizing  the  total  return. 
The  recursion  equations  for  the  various  phases  are  defined  as  follows: 

2.1.1)  For  the  Diverging  Branch  (from  stage  11  to  stage  Ml) 
fu(xu)  -  .ax  ru<xu,  du) 

1  i  dll  i  P11 

£1(x1)  '  “*  d„l>  +  Vl.l'V/Vl' 


where 


1  <  d  .  <  P  . 

—  ml  —  ml 

m  =  2 ,  3 ,  ....  M 


Using  the  above  equations,  the  optimal  branch  return  and  optimal  decisions 
are  computed  for  each  possible  value  of  x^. 

2.1.2)  For  the  Main  Serial  Process  (from  stage  1  to  stage  S-l, 

(prior  to  junction  node) 

£1'X1^  =  max  rl^xl’  dl^ 

1  _<  djL  £  PI 

F  (x  )  =  max  [r  (x  ,  d  )  +  f  . (t  (x  ,  d  ))] 
n  n  n  n  n  n-i  n  n  n 

1  <  d  <  P 
—  n  —  n 

where  n  =  1,  2,  ...,  S-l 

The  optimal  return  f  (x  )and  optimal  decision  d  at  each  stage  are  saved 

n  n  n 

for  each  possible  input  value  x^i 

2.1.3)  For  the  Stage  S  (junction) 

fS+Ml^XS^  “  m3X  ^rS(xS’  V  +  fS-l^tS^xS’  +  fMl^tSl^rS’  dS^ 

1  i  ds  1  ps 
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At  this  stage,  the  optimal  return  ^S+I.^(xg)  *s  tlie  combination  of  the 

optimal  return  at  stage  S,  rg(xs>  dg)  the  optimal  return  from  the  main  serial 

process  preceding  stage  S,  fg_^(tg(xs>  d^)  and  the  optimal  return  from  the 

branch,  f  (t C1 (x  ,  d  ).  For  each  possible  value  x_,  f„,„, (x_)  and  d.  are  reserved 
I'll  bibb  S  S+Ml  S  S 

at  this  stage. 

2.1.4)  For  the  Remaining  Stages  (from  stage  S  +  1  to  stage  N,  the  terminal 
node) 

The  optimal  return  at  each  remaining  stage  from  S  +  1  to  N  can  be  obtained 
as  in  the  usual  serial  systems,  i.e.. 


fN+Ml(xNJ  maX  ^rN(xN’  djp  +  fN-l+Ml(tN(xN’  dN^ 


1  <  d  <  P 
-  N  ~  N 

where  m  =  S  +  1,  . . .  ,  N 

2.1.5)  Determination  of  the  Optimal  Decision  and  Return  at  Each  Stage 

•k 

At  the  final  stage  N,  the  optimal  input  to  the  system  can  be  obtained  by 

letting 


WV  =  raax  {fN+Ml(xK)} 


With  the  optimal  input  x%T  and  optimal  decision  d__  obtained  from  a  decision  table, 

N  N 

*  * 

we  can  produce  optimal  stage  return  rVT  and  optimal  stage  output  xXT  -  as  follows: 

N  N~1 

‘K  *  rS<V  V 

*N-1  " 

This  process  continues  from  stage  N  down  to  stage  S  +  1. 

*  * 

At  junction  stage  (stage  S) ,  the  optimal  stage  input  x  and  stage  decision  d  ,  the 

w  b 

* 

optimal  branch  input  x^  are  obtained  via  the  transition  equation 

’Sll  =  tsi(xs>  ds)' 

For  the  remaining  processes,  the  stage  transformation,  return  function,  and  decision 
tables  can  be  used  at  each  stage. 
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2.1.6)  Input  Data  Required  for  the  Algorithm 

The  input  data  for  the  algorithm  are  as  follows: 

N  *■  #  of  stages  in  the  main  serial  process 

M  =  it  of  stages  in  the  branch 

S  =  junction  stage 

K  =  upper  bound  in  the  input  value  x  ,  ,  m  =  1,  2,  ....  M 

ml  ml 

2.1.7)  Output  List  of  the  Algorithm 

Basically,  the  output  of  this  algorithm  consists  of  the  following: 

1.  Return  table  at  each  stage  for  each  input  value. 

2.  Decision  table  at  each  stage  for  each  input  value. 

3.  Optimal  input,  decision,  and  return  at  each  stage. 

In  the  next  section,  we  present  a  companion  flow  chart  for  a  computer  program 
to  implement  the  basic  algorithm  whose  steps  were  detailed  above.  In  Section  2.3,  a 
simple  problem  is  posed  and  solved.  The  situation  involving  a  set  of  constraints 
is  next  treated. 

In  Section  2.4,  the  diverging  branch  problem  is  analyzed  in  terms  of  the 
algorithm's  sensitivity  to  a  variation  in  N,  the  number  of  stages,  the  complexity 
of  branches,  as  well  as  the  transition  functions. 

2.2  The  Algorithm:  Special  Structure  and  Flowchart 

The  high  level  algorithm  developed  above  and  flowcharted  in  the  sequel,  although 
akin  to  the  conventional  dynamic  programming  version,  has  some  special  structures 
worthy  of  note.  A  direct  application  of  the  usual  approaches  would  dictate  enormous 
storage  requirements  when  processing  nonserial  networks,  thus  making  the  processing 
of  large  networks  virtually  impossible.  To  mitigate  this  problem,  we  devise  a 
technique  which  enables  us  to  indicate  the  optimal  decision  values  at  each  stage  by 
appending  =  1  +  K  to  the  state  entry  in  the  corresponding  transaction  table,  where 
K  =  max  {K^,  ...»  1^,  ....  }  .  Although  this  adds  one  more  state 

variable  and  this  stage,  it  enables  us  to  eliminate  the  storage  requirement  for 
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the  optimal  decision  which  is  normally  the  case  with  classical  algorithms. 

Any  future  reference  to  this  table  entry  is  made  as  the  entry  Mod  K^.  When 

needed  later,  the  optimal  decision  values  can  be  retrieved  by  searching  only  one 

row  of  the  table  for  the  entry  value  greater  than  or  equal  to  K,. 

d 

The  computer  algorithms  developed  for  the  diverging  branch  system,  as 
well  as  for  other  systems  in  this  report,  are  done  in  Fortran.  They  are  based 
on  the  analysis  presented  in  the  foregoing  sections  and  flowcharted  in  Fig.  2.2. 
The  input  data  dictated  by  the  dynamic  programming  constructs  are  in  terms  of 
functions  of  the  respective  variables.  This  also  mrkes  it  possible  to  handle 
larger  problems  without  the  forbidding  storage  limitations. 


2. 3  Example  (A  Diverging  Branch  System) 

Let  us  explicate  the  steps  of  this  algorithm  by  considering  an  example. 

Suppose  it  is  desired  to  maximize  the  sum  of  stage  returns  as  a  function  of  the  input 
x^  for  the  problem  with  the  structure  given  in  Fig.  2.3  and  the  data  in  Table  2.1. 


Figure  2.3:  A  Diverging  Branch  System  Example 
Table  2.1:  The  Return  and  the  Transformation  Function  in  the  System. 


Stage 

Decision 

Return 

Transformation 

Constraints 

2 

1 

dl 

rl  =dl 

dl  E  [-i’  di] 

2 

d2 

r2  +  d2 2 

X1  =  X2  +  d2 

Xi  e  ^-i’  Xi^ 

3 

d3 

r3  =  d32 

2 

X2  =  x3  *  d3 

X21  =  X3  +  d3 

- 

11 

dll 

ru  *  du 

dil  E  l-il’  dilJ 

21 

d21 

,  2 
r  ■»  d 

21  °11 

X11  =  X21  +  d21 

*ii  c  'Su-  ;ii> 

4 

d4 

r4  "  d42 

x3  '  *4  +  \ 

- 

5 

d5 

r5  =  d52 

X.  *  X,  +  xc 

4  4  5 

- 

*f*i(3i>,  xi 
each  corres 

C*l>-  *  a”<1  iii<*u 

ponding  variable. 

)  represent  the  lower 

21 

bound  (upper  bound)  of 

In  this  system,  the  stages  11  to  21  and  1  to  2  can  be  optimized  using  usual 

recursive  procedures  as  two  disjoint  serial  systems. 

At  stage  11  and  the  diverging  branch,  we  have 

2 

f  (x  )  =  max  d 
11 v IV  11 

dn 


and  the  optimal  decision  is  found  to  be 

^l^ll*  =  dll 

Also,  at  stage  21,  we  see  that 

d21(x21^  =  d21 


with 


f21(x21^  =  d21  +  d21  * 


For  stages  1  and  2  of  the  main  serial  process,  the  optimal  decision  and  the 
value  of  are  £iven  by 


with 


W  *  dl 

d2(x2)  =  d2 
f2(X25  “  ^±2  +  *2 2 ' 


Now,  at  stage  3  the  two  optimal  returns  ^21^21^  and  ^2^*2^  arecom^ine<^  with  the 
stage  3  return  as  follows: 

f3(x3)  =  max  [d32  +  (d^  +  d^  )+  (d^  +  d^)] 


We  again  observe  that 


with 


d3(x3)  -  d3 

f3(x3}  =  di2  +  l2  +  d32  +  dll  +  d21 


The  optimization  of  the  remaining  stages,  from  stage  4  to  5,  can  also  be 
carried  out  recursively. 

We  finally  have  the  following  optimal  return  function  at  stage  5. 

5  _  2  ^  -  2 

f  (X  )  =  1  di  +  *  diL 

i“l  i»l 
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The  optimal  solution  can  be  summarized  as  follows  (Table  2.2) 


Table  2.2:  Optimal  Solution  to  the  System  Posed  in  the  Example 


Stage 

Optimal  Input 

Optimal  Decision 

Optimal  Return 

1 

X1  =  x2  +  dx 

dl  =  di 

rl  =  dl2 

2 

x2  =  x3  +  d3 

d2  =  d2 

r2  =  d22 

3 

X3  =  X4  +  54 

d3  ‘  d3 

r3  =  d32 

11 

X11  =  X21+  55 

dll  =  511 

ru  ■  5112 

21 

x2i=  x3  +  d3 

d21  =  d21 

3  2 

r21  d21 

4 

X4  =  X5  +  d5 

d4  =  d4 

r4  =  d42 

5 

X5 

d5  “  d5 

r5  =  d52 

We  will  now  solve  the  diverging  branch  system  defined  in  Table  2.2  and 
Table  2.1  using  the  computer  algorithm  given  in  Section  2.2. 

Some  constraints  on  the  input  and  the  decision  variables  are  added  as 
in  the  following  problem. 


5 

2 

max 

l 

n=*l 

r  +  l  r 

n  i 

m=l 

ml 

8  •  t  • 

1 

5  d  S3 
n 

n 

=  i, 

..  ,  5 

1 

S  d  ,  s  3 
ml 

m 

=  i. 

2 

1 

S  x  S  20 
n 

n 

=  2, 

....  5 

1 

S  x^  S  5 

1 

<  x  ,  S  20 
ml 

m 

=  1, 

2 
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The  return  and  decision  tables  at  each  stage  for  each  input  are 
shown  in  the  computer  output  attached  (See  Table  2.3).  The  optimal  input. 


decision, and  return  at  each  stage  are  also  shown  in  the  output. 

2.4  Analysis  of  the  Diverging  Branch  System 

In  Section  2.1,  we  had  assumed  that  the  input  and  decision  variables  at 
each  stage  had  the  following  integer  values : 


.  ±Kil 

i  =  1, 

2,  ...,  M 

±Ki 

i  =  1, 

2 . N 

,ipil 

i  =  1, 

2 . M 

<  p. 

—  X 

i  -  1, 

2 . N 

Let  K  =  max  (K.^,...,  ;  K . 1^) 

Now,  we  will  discuss  the  effects  on  the  storage  and  the  computer  time 
of  our  algorithm. 

First,  let  us  define  the  storage  requirement  of  the  diverging  branch  algorithm 
as  a  function  of  K,  as  follows: 

0(K)  =  (M  +  N  +  3)  K 

The  above  storage  requirement  can  be  easily  verified  from  the  algorithm  [16]. 


2.4.1)  Sensitivity  to  N 

Let  us  assume  that  the  maximum  value  K  remains  unchanged  with  the  increase  of 
the  number  of  stages  N,  in  the  main  serial  process.  The  storage  requirement 
in  this  case  increases  by  K  with  each  additional  increase  in  the  number  of  N. 
Moreover,  the  requirement  0(K)  is  approximately  proportional  to  N  if  the  number 
of  stages  in  a  diverging  branch,  M  is  relatively  small. 

Table  2.4  shows  the  computational  results  of  the  example  in  Section  2.3 
with  the  increasing  number  of  stages. 
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Table  2.4:  Computer  Storage  Requirements  of  the  Diverging  Branch  Problem 


Number  of  Stages 

Storage  Requirement 

CPU  Time 

N 

0(K) 

(Seconds) 

5 

200 

.279 

10 

300 

.371 

15 

400 

.480 

As  shown  in  the  above  table,  both  the  storage  requirement  and  the  CPU  time 
seem  to  increase  linearly  with  the  increase  of  N. 

2.4.2)  Sensitivity  to  the  Complexity  of  Branches 

When  the  number  of  diverging  branches  increases,  the  storage  requirement 
is  not  so  simple  as  in  the  previous  case.  Consider  a  multi-diverging  branch 
system  in  Fig.  2.4  where  the  number  of  branches  is  D  and  each  branch  has 
M^.i  =  1,  2,...,  D  stages.  Whenever  a  diverging  branch  is  added,  the  branch  needs 
storage  both  for  the  branch  return  and  for  the  optimal  decision  at  each  stage  of 
the  branch,  hence,  the  storage  requirement  increases  by  (1L  +  1)  K,  i  =  1,  2, . . . ,  D. 
Thus,  the  total  storage  requirement  for  the  D  different  diverging  branches  can  be 
represented  as  follows: 

D 

0(K)  =  (  E  (M  +  1)  +  N  +  2)  K. 
i=l  1 

2.4.3)  Sensitivity  to  the  Complexity  of  Transition  Functions. 

When  the  transition  functions  t  ,  t  ,,  and  t  ,  can  be  represented  as  a  simple 

n  mi  S 1 

linear  combination  of  stage  input  x  (or  x  ,)  and  decision  d  (or  d  ,),  the  storage 

n  mi  n  mi 

requirement  is  not  complex.  However,  if  a  system  requires  a  transition  function 

which  has  nonlinear  term  of  x  or  d  ,  the  storage  problem  becomes  serious. 

n  n 
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i 

i 

i 

i 

i 
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Consider  the  example  in  Section  2.3  if  the  transition  function  is  given  by 

2 

t  (x  ,  d  )  =  x  +  d  for  n  =  1,  2 . 5. 

-  n  n  n  n  n 

then,  K  =  max  K^,  1  <  i  <  5  becomes  a  huge  number  in  an  unconstrained  problem. 

As  a  result,  the  computational  time  will  be  increased  dramatically. 

2.4.4)  Sensitivity  to  the  Complexity  of  Return  Functions 

Provided  that  we  use  a  function  of  x  and  d  for  the  return  at  each  stage, 

n  n 

the  complexity  of  return  function  does  not  affect  the  storage  requirement  of  this 
algorithm.  This  is  because  the  storage  requirement  is  strictly  a  function  of 
K,  N  and  M. . 

l 


T* 

■ 


i 

i 

i 
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Chapter  3 


ANALYSIS  OF  CONVERGING  BRANCH  NONSERIAL  NETWORKS 


3.1  Development  of  a  DP  Algorithm  for  the  Converging  Branch  System 

Let  us  direct  our  attention  to  the  converse  of  the  diverging  branch  system, 
namely  a  converging  branch  nonserial  network.  In  its  simplest  form,  a  number  of 
parallel  serial  systems  join  together  at  a  junction  node  and  then  feed  their 
outputs  to  a  serial  system.  A  simple  example  consisting  of  two  input  parallel 
branches  and  one  serial  output  is  exhibited  in  Fig.  3.1. 

For  analysis  and  algorithm  development,  consider  this  structure  as  a  main 
serial  system  n,  =  1,  2,  ...,  N  and  a  branch  n,  =  1,  2,  ...,  M.  The  convergence 
occurs  at  node  (stage)  S.  The  transformation  at  this  stage  may  be  written  as: 


S-l 


cs(xoi’  V  dS) 


The  transition  function  for  the  other  stages  may  be  represented  as  in  the  usual 
serial  processes  as  follows: 


For  the  Branches 


For  the  Main 


m-1,1 


n-1 


tml^Xml ’  dml) 


t  (x  ,  d  ) 
n  n  n 


m  =  1 ,  2 ,  ...»  M 


n  =  1 ,  2 ,  .  .  .  ,  N 
n  5*  S 


We  define  the  returns  for  each  stage  similarly.  Thus, 


rs  = 

rs(xoi’  xs’  ds) 

r  = 

r  (x  ,  d  ) 

•H 

II 

C 

2,  .. 

n 

n  n  n 

rml 

=  r  (x  , ,  d  ) 
ml  ml  ml 

B 

ll 

h-* 

2,  .. 

To  develop  the  algorithm,  we  proceed  as  follows.  We  first  decompose  the  system 
into  three  components  corresponding  to  stages  11,  21  to  Ml,  and  1  to  N.  For  stages 
1  to  N,  we  separately  consider  stages  1  to  S  -  1,  S,  and  finally  S  +  1  to  N.  To  find 
the  optimaL  branch  return  f  (x  )  we  will  use  the  backward  recursion.  Next  we  will 

4ii  UI 
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maximize  (x„.)  over  x^,,  .  The  recursion  equations  for  the  different  phases 
Ml  01  711 

may  then  be  defined  as  follows:  (Section  3.1.1) 


Figure  3.1:  A  Converging  Branch  System 


3.1-1)  For  Stage  11 
We  solve  the  problem 

fn(xir  xoi)  =  max  rn(xir  du) 

1  dn  pn 

s.t.  xQ1  =  t11<*11»  dn> 

In  ocher  words,  for  each  input  value  x^  we  will  find  the  optimal  decision  d^ 
which  satisfies  xQ1  =  tu(xu,  du)  and  also  maximizes  the  stage  return.  For  each 
value  of  (xix,  xQ1)  ,  the  optimal  decision  dn  and  optimal  return  ru  are  saved. 
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At  each  stage  from  21  to  Ml,  the  optimal  decision  d  .  and  optimal  return 

ml 

are  computed  for  each  pair  of  (x  Xq,  ) .  At  stage  Ml,  f  (  x,^,  xqj)  is 
found  and  the  value  of  x^  which  maximizes  the  branch  return  for  each  value 
of  is  obtained. 


3.1.3)  For  the  Main  Serial  Process 

3.1.3-i)  The  optimal  return  from  stages  1  to  S  -  lean  be  found  by  using  the 
usual  recursive  procedure,  i.e., 

fi(x1)  =  max  r1(x1> 


f  (x  )  =  max  [r  (x  , 
n  n  n  n 


d  )  +  f  . 
n  n-1 


(t  (x  ,  d  ))] 
n  n  n 


1  <  d  <  P, 
n  1 

where  n  =  2,  3,...,  S  -  1 

3.1.3-ii)  At  Stage  S  (the  junction  node) 

The  optimal  branch  return  f^Cx^,  xQ1)  is  combined  with  the 
return  at  stage  S  and  the  optimal  return  from  stages  1  through  S  -  1  using  the  recursion 
equation 


f$(xs)  =  max  [rs(*01,  xg,  dg)  +  ^(tg^.  xg,  dg)> 


+  fMl(xMl*  X01)] 


where  the  maximization  is  over  1  £  Xq^  anc*  ^  In  ot^er  worc^s> 

at  junction  S,  we  compute  the  optimal  return  fg(xg)  and  determine  optimal  branch 


output  xq^ >  and  optimal  decision  dg,  for  each  input  value  of  x,,.  We  can  also 

*M1 


01-  •  -  -S>  - "S' 

obtain  the  optimal  branch  input  x  which  maximizes  the  branch  return  using  the 


value  of  x0^. 

3.1.3-iii)  For  Stage  S  +  1  to  N 

The  recursion  equation  is  given  by 

W  =  max  [rN(xN,  dj.)  +  ^(t^,  dj)] 


1  <  d  s  Ps 
n  =  S  +  1,...,  N 
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where 


At  the  final  stage  N  the  optimal  system  return  for  each  input  value  of  x  can 

n 

be  obtained. 

3.1.4)  Determination  of  the  Optimal  Decision  and  Return  at  Each  Stage 

is 

At  the  final  stage,  the  optimal  input  x^  can  be  obtained  which  maximizes 

■flf 

with  the  optimal  decision  d^  obtained  from  the  decision  table.  We  will 

* 

proceed  from  stage  N  -  1  to  stage  S  +  1.  At  stage  S,  the  optimal  input  x,,  and 

* 

optimal  branch  input  x^^  are  found  as  follows: 

* 

Xs  =  CS+1^XS+-1  ’  ds-fl^ 

*  * 

Now  that  Xg  has  been  found,  the  optimal  branch  input  x^  can  be  obtained.  This  is 

because  we  decided  optimal  for  each  value  of  Xg  when  evaluating  the  optimal 

objective  function  value  at  stage  S.  For  the  remaining  stages  the  optimal  stage 

input  and  decision  can  be  obtained  using, the  stage  transformation  function: 

XN  =  CN+1(XN+1’  dN+l)*  N=>S_1»S~2»--*»1 
and  the  decision  table,  respectively. 

3.1.5)  Input  Data  Required  for  the  Algorithm 

The  algorithm,  akin  to  that  developed  for  the  diverging  branch  system,  is 
designed  to  receive  the  following  input  specifications  in  Fortran: 

H  =  it  of  stages  in  the  main  serial  process 
M  =  it  of  stages  in  the  converging  branch 
S  =  junction  stage 

=  upperbound  of  the  input  value  x^ ,  i  =  1,  2,  ...,  M 
®  upperbound  of  the  input  value  x^,  i  =  1,  2,  ...,  N 
Pil  =  uPPerb°und  of  the  decision  value  d^,  i  =  1,  2,  ...,  M 
*  upperbound  of  the  decision  value  d^  1,  2,  ...,  N 

3.1.6)  Output  List  of  the  Algorithm 

At  the  completion  of  the  operations,  unless  otherwise  specified,  the  algorithm 
outputs  the  following: 
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1.  Return  and  decision  table  for  each  pair  of  (x  .  ,  x..),  m  =  1,  2,  M 

mi  ul 

2.  Optimal  branch  input  and  branch  return  for  each  value  of  x^. 

3.  Decision  table  for  main  serial  process. 

4.  Optimal  branch  output  x  ^  for  each  value  of  junction  input  x^. 

5.  Optimal  input,  decision  and  return  at  each  stage. 

3.2  The  Algorithm,  Flowchart,  and  Structure 

As  is  evident  from  the  analysis  of  the  foregoing  section,  the  computational 
schema  for  the  converging  branch  system  differs  from  that  developed  for  the  diverging 
branch  system.  The  flow  chart  for  the  converging  branch  system  is  given  in  Fig.  3.2. 
Although  there  are  similarities  in  the  logic  of  the  flow  charts,  major  differences 
occur  in  the  optimization  procedure  as  well  as  the  difficulty  of  performing  the 
optimization. 

Comparing  Figs.  2.2  and  3.2,  we  notice  that  the  first  major  difference  arises 

in  section  A  where  instead  of  storing  the  optimal  return  and  decision  at  stage  Ml, 

a  vector  optimization  involving  two  state  variables  x^  and  xQ1  is  performed  in  the 

converging  branch  case.  The  differences  surface  again  in  the  computation  of 

f  ... (x  ).  From  then  on,  the  charts  follow  basically  the  same  procedure. 
n+Mi  n 

3. 3  An  Example  Involving  the  Converging  Branch  Algorithm 
Let  us  now  demonstrate  the  use  of  the  algorithm  developed  in  section  3.1  in 

the  solution  of  a  converging  system  problem.  Suppose  it  is  desired  to  find  the 
policy  which  maximizes  the  sum  of  the  stage  returns  as  a  function  of  the  inputs  x^ 
and  ^or  the  Problem  with  the  structure  depicted  in  Fig.  3.3  using  the  data  of 
fable  3.1.  In  this  problem,  the  serial  component  consists  of  stages  1  through  3 
while  the  converging  branch  consists  of  11  and  21  with  convergence  occurring  at 


node  3. 


Start 


f„l<x.l-  V  ■  *“  dml  ^ 1 

1  S  d„l  S  P„1 


*01>  '  trn,l<V’  dml> 


,'t-  *01  "  S>i(x.i’  d„i> 


1id»liP„l  +  <V» 


m  ®  m  +  1 


rn  *=  M 


m 


:o: 


Figure  3.3:  A  Converging  Branch  System  Example 


Table  3.1:  The  Return  and  Transformation  Function  in  the  System 


We  will  first  begin  with  the  branch  system  and  determine  ^21^21’  XQ1^‘ 


At  stage  11, 


'n^ir  V  '  "*  dif 


s.t.  x01  -  xu  +  cu 

Substituting  the  constraint  into  the  recursion  equation,  we  have 

fii(xn*  X01)  =  (x01  ”  X11J 

At  stage  2,  the  recursion  equation  is  given  by 

f21(x21*  X01}  =  max[d21  +  (X01  “  X21  “  d21)2] 
d21 

Since  ^21^x21’  X01^  ds  3  convex  function  of  the  decision  variable  d  it  is  easy 
to  see  that  the  optimal  decision  variable  with  the  value  of  ^l^Jl'  X01^  is 
given  by 

d21(x21’  X01}  =  d21 

d21(x21’  X01)  =  2d21  +  2(x21  "  *01*  d21  +  (x21  '  X01) 

Now,  the  main  serial  system  through  stage  2  is  optimized  in  the  usual 

way  to  obtain  f2^X2^'  At  stage  1,  we  have 

2 

^1^X1^  =  max  dl 

and  the  optimal  decision  is  given  by 

dl^Xl^  “  dl* 

By  proceeding  as  at  stage  1,  we  have 
d2(x2)  -  d2 
f2(x2)  -  d:2  +  d22 

At  stage  3,  the  branch  return  is  combined  with  the  serial  return  from 
stage  1  to  stage  2  using  the  recursion  equation 

f3+21(x3’  X21^  =  max ^d3  +  ^dl  +  d2  )  +  f2d21  +  2(x2l”  X01^  d21 


+  (x21  -  xQ1)  }] 


where  the  maximization  is  taken  over  x^^  and  d^. 


38 


The  optimal  value  of  Xq^  Is  found  to  be 

X01(V  X21)  =  X01 

with  d3^x3’  x21^  =  d3 


and  f3+21(*3'  *21)  "  5 1  *  l2  +  *  2d2?  +  2(*21  ‘  ;0l)d21  + 

‘*21  “  X01>2 

Having  combined  the  branch  optimally  with  the  main  serial  systems  we  accomplish 
the  analysis  of  stage  4  and  5  with  the  standard  backward  recursion.  Then  we 
finally  have 

W  =  d4 

d5(xj  =  d5 

f5+21(x5’  X21J  =i^1  di  +  2^21  +  2(x21  "  X01)d21  +  (x2l  "  X01) 
Retracing  our  analysis,  we  find  the  remainder  of  the  optimal  solution,  which 
is  summarized  in  Table  3.2  as  follows: 

Table  3.2:  Optimal  Solution  to  the  System 


Stage 


Optimal 

Input 


Optimal 

Decision 


Optimal 

Return 


1 

X1  " 

X-  + 
0 

X21+  d2  +  d3  +  d4  +  d5  + 

dn  +  dn 

dl  =  ai 

^  =  ^I2 

2 

X2  = 

X..  + 
0 

X21  +  d3  +  54  +  d5  +  dll 

+  d21 

d2  =  d2 

r2  =  d22 

3 

X3  “ 

X.  + 
0 

54  +  S 

d3  =  d3 

r3  "  d32 

X01 

s  X 

21 

+  d21  +  dlJ 

- 

- 

11 

X11 

"  X21 

+  54 

dll  =  dll 

ru  ’  dii 

21 

X21 

d21  =  d21 

r21  ‘  d21 

4 

X4  * 

X5  + 

55 

d4  =  d4 

r4  =  d42 

5 

X. 

0 

d5  ‘  d5 

r5  "  d52 

Let  us  now  solve  the  converging  branch  system  defined  in  Fig.  3.3  and  Table  3.1 
using  the  computer  algorithm  given  in  Section  3.2.  We  will  add  some  constraints 
on  the  variables  as  in  the  following  problem: 

5  2 

max  E  r  +  E  r  , 


n=l 

n  , 

m=l 

ml 

1  < 

d  <  3 
n 

n  = 

i. 

1  < 

dml  *  3 

m  * 

i, 

1  < 

X  <30 
n 

n  = 

i, 

1  < 

Xi  ~ 5 

1  < 

Xml  S  10 

m  = 

0 

The  computational  results  are  shown  in  the  computer  output  of  Table  3.3.  The  last 
table  shows  the  optimal  decision  and  return  at  each  stage. 

3. 4  Analysis  of  the  Converging  Branch  System 

Different  from  the  diverging  branch  system  the  storage  requirement  of  the 
converging  branch  system  depends  not  only  on  the  value  K  but  on  Kni  and  K-  . 

U-L  W 

these  are  defined  as  follows 

K  =  max  (Kj^  ,  K21,...,  Kj^,  Kx . 1^) 

1  5  xoi  s  Koi 

and 

The  storage  requirement  for  the  process  of  main  serial  is  (N  +  2)K  ,  (N  x  K)  for 
the  decision  value  and  (2  x  K) for  the  stage  return  as  discussed  in  the  diverging 
branch  system.  See  Section  2.4. 
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However,  the  storage  problem  for  the  converging  branch  is  a  little 


complicated.  It  can  be  analyzed  as  in  the  following  cases. 

1.  At  each  stage  (11,  21,...,  Ml)  for  each  pair  of  input 
to  the  stage  and  the  branch  output  (x  ) ,  we  deter¬ 
mine  the  decision  value  which  optimizes  the  branch 
return.  Hence,  the  algorithm  needs  2  X  K  X 

storage  for  computing  the  total  branch  return  for 
each  pair  of  branch  input  and  branch  output  x  . 

Another  M  X  K  X  storage  is  required  for  the 

decision  value  at  each  stage.  Therefore,  it  becomes 
(M  +  2)  X  K  X  K  . 

2.  Storage  requirements  for  the  decision  of  optimum 
branch  input  and  branch  return  for  each  value  of 
branch  output  x^  becomes  2  X  K^. 

3.  At  junction  stage  S,  we  need  to  determine  the  optimal 
value  of  the  input  from  the  converging  branch  for 
each  possible  value  of  the  input  from  stage  S  +1  to  S. 

This  requirement  becomes  K^. 

From  the  above  analysis,  the  total  storage  requirement  of  the  converging  branch 
algorithm  can  be  represented  as  a  function  of  K.  s  id  Kg  as  follows: 

0(K,  KQ1,  Kg)  =  (N  +  2)K  +  (M  +  2)  X  K  X  KQ1  +  2Kq1  +  Kg 
If  we  assume  the  levels  of  the  state  variables  x^  and  x<.  to  be  equal  to  K,  then 
the  above  expression  becomes 

0(K)  =  (M  +  2)  K2  +  (N  +  5)K 

Next,  we  will  discuss  the  effects  of  storage,  and  time  for  the  following 
cases. 


3.4.1)  Sensitivity  to  N 

As  shown  in  the  function  0(K),  the  storage  requirement  increases  by  K 
as  we  increase  one  stage  of  the  main  serial  process.  This  is  the  same  as  in 
the  diverging  branch  system. 
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However,  notice  that  the  storage  requirement  in  the  converging  branch 

is  highly  affected  by  the  number  of  stages  in  the  branch.  This  is  because 

2 

each  additional  increase  of  M  requires  a  storage  of  the  order  K  . 

The  computational  results  with  increasing  number  of  N  in  the  example 
in  Section  3.3  are  summarized  in  the  following  table: 

Table  3.4:  Computational  Experience  with  the  Converging  Branch  System 


Number  of  Stages 

N 

Storage  Requirements 

0(K) 

CPU  Time 
(Seconds) 

5 

1440 

.496 

10 

1590 

.695 

15 

1740 

.841 

3.4.2)  Sensitivity  to 

the  Complexity  of  Branches 

As  discussed  before,  the  storage  requirement  in  a  converging  branch 
system  is  highly  affected  by  the  number  of  stages  in  a  converging  branch. 

Moreover,  if  the  number  of  branches  increases,  the  problem  becomes  really 
serious. 

Consider  the  multi-converging  branch  system  shown  in  Fig.  3.4,  where  the 

number  of  converging  branches  is  D  and  each  branch  has  M^,  i  =  1,  2,...,  D  stages  in 

it.  When  a  converging  branch  is  added,  the  branch  needs  storage  of  the  order  of 
2 

(M^  +  2)K  +  3K  as  analyzed  before. 

Hence,  the  storage  requirement  for  the  above  multi-converging  branch 
D  2 

system  can  be  0(K)  =  (E  M.  +  2)K  +  (N  +  2  +  3D)K. 

i=l  1 

3.4.3)  Sensitivity  to  the  Complexity  of  Transition  functions 
As  we  have  discussed  in  the  diverging  branch  system,  the  complexity  of 
transition  function  affects  the  levels  of  discretization  of  the  input  state 
at  each  stage.  Hence,  it  directly  affects  the  maximum  number  K  on  which  the 
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storage  requirements  mainly  depends. 

3.4.4)  Sensitivity  to  the  Complexity  of  Return  Functions 

Again,  the  return  function  has  no  impact  on  the  storage  requirement  and 

computational  time  providing  the  algorithm  employs  a  function  in  representing 

the  input  variables  x  and  decision  d  . 

n  n 

3.5  A  Modified  Converging  Branch  Algorithm 

The  algorithm  developed  in  Section  3.1  and  used  to  solve  the  sample  problem 
in  Section  3.3  is  somewhat  restricted  in  the  classes  of  problems  it  solves.  To 
use  it  in  solving  problems  involving  complex  functions,  especially  with  regards  to 
the  transition  and  return  functions,  a  modification  was  necessary.  This  led  to  the 
development  of  the  modified  algorithm.  Let  us  then  illustrate  the  use  of  the  modi¬ 
fied  converging  branch  algorithm  with  the  following  example  problem.  Suppose  it 
is  desired  to  find  the  policy'  which  minimizes  the  sum  of  the  stage  returns  as  a 
function  of  the  inputs  and  f°r  t*1e  Pr°blem  with  the  structure  given  in 

Fig.  3.5  using  the  data  given  in  Table  3.5.  First  notice  the  presence  of  fractional 

numbers  in  the  return  functions  for  each  stage.  Since  there  are  no  stages  after  the 
converging  stage  in  the  main  serial  system,  we  begin  with  the  two  stage  branch  system, 
and  determine  ^l^^l’  X01^ '  sta8e  H 

2 

fll^Xll’  X01^  =  min  0,2  dll 
dll 

s.t.  x01  =  xx  -  du 
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Substituting  the  constraint  into  the  recursion  equation, we  have 

fll^Xll’  X01)  °'2  *X1  "  x01^ 

At  stage  21,  we  see  that 

f21(x2l’  X01}  =  “ln  075  '  10d21  +  0>2  (X21  +  d21  "  X012)> 
d21 

Differentiating  the  term  in  brackets  with  respect  to  d^  and  setting  the 
derivative  to  zero  yields 

d21(x21’  x01)  =  25  +  X01  “  X21 

and  f21^X21*  X01^  =  250  ”  10x01  +  10x21 

Now,  at  stage  1  the  optimal  branch  return  is  absorbed.  Thus, 

fl+21^xl’  X21^  =  min  dl2  +  250  “  10x01  +  10x21^ 
x01’  dl 

S.t.  dj  -  Xx  *  X01 

Substituting  the  constraint  into  the  foregoing  recursion  equation,  we  obtain 
fl+21^Xl*  X21^  =  mln  ^*1^xi  +  xoi^2  +  250  ~  10x01  +  10x21^ 


! ' 


The  optimal  value  of  is  readily  found  to  be 

xoi(xr  V  =  50  ~  xi 

with  d  (x  ,  x21)  =  50 

and  fl+21^xl*  x21^  "  10  (xj.  +  x2i^ 

At  stage  2,  we  have 

f2+21^x2’  X21^  "  min  ^60°  -  100x2  +  d2  +  10^x2  +  d2  +  x21^ 
d2 

The  optimal  value  of  d2  in  the  above  recursion  equation  is  independent  of  the 
stage  inputs  x2  and  x^.  So, if  we  assume  the  feasible  region  of  d2  as 

d2  E  ^-2*  d2^ 
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then,  we  have  the  optimal  value  of  d2  as 

d2  =  -2 


and 


f2+21^X2’  X21^  =  600  +  i;i-2  =  90x2  +  10x21 


Finally  at  stage  3,  the  recursion  equation  becomes 


f3+21^X3’  X21^  =  min  ^°'1  ^50  -  d3^  +  600  +  lld2-  90  ^x3  +  d3^ 

d3  +  10*21> 

It  is  easy  to  see  that  =  500  is  the  solution  with 

f3+21^X3*  X21^  =  24150  +  -2  “  90x3  +  10x2l‘ 

Retracing  our  analysis,  we  find  the  remainder  of  the  optimal  solution 

which  is  summarized  as  follows: 


Table  3.6:  Optimal  Solution  to  the  System  of  Fig.  3.5  and  Table  3.5 


Stage 

Optimal  Input 

Optimal  Decision 

Optimal  Return 

1 

x^  =  +  500  +  d2 

d1  =  50 

250 

X01  =  X3  -  300  '  -2 

- 

- 

11 

X11  =  X3  "  425  =  -2 

dll  "  25 

125 

21 

X21 

d21  =  x3  ‘  X21  '  425 

-  d2  10x3+  10x21  +  4625  • 

2 

x2  »  x^  +  500 

d2  =  -2 

-100x3  =  49400  -  d2 

3 

X3 

d3  -  500 

20250 

Let  us  next  solve  the  same  problem  described  in  Table  3.5  with  the  following 
constraints  added  to  the  input  and  decision  variables  at  each  stage 


3  2 

min  I  r  +  Z  r  . 
•  n  «  tn. 

n~l  ra*l 

s. t.  491 <  x^  £  510 

491  S  Xj  s  510 
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I  a 


0  £  x3  s  5 
-464  s  xQ1  s  445 

-439  <5  xxl  S  -420 

0  S  x2i  s  5 

41  ^  d1  s  60 

0  S  d2  s  5 

11  <;  d3  S  30 

11  £  dn  £  30 

-439  <;  d2  £  -420 

xn’  V  dn'  d»l  «  T-  "  -  }-  ?•  3 

m  =  1,  2 

where  T  is  the  set  of  constraints  in  Table  3.5.  Notice  that  this  is  akin  to  the 
problem  solved  earlier  in  that  the  objective  function  is  similar.  However,  the 
stage  return  functions  are  different.  The  solution  is  effected  using  a  modifi¬ 
cation  of  the  algorithm  developed  earlier.  The  computational  result  is  shown 
with  the  computer  output  in  Table  3.7  .  Note  also  that  from  Table  3.5,  the  total 
optimal  return  of  the  system  in  Fig.  3.5  becomes 
10x21  _  90x3  +  U-2  “  23150. 

With  the  constraints  given  in  the  above  problem,  the  optimal  solution  is  obtained 
when 

X21  =  °»  x3  =  5»  and  -2  ” 

Further,  note  that  in  the  computer  output  the  problem  was  solved  by  changing 

the  minimization  of  the  objective  function  to  the  maximization  of  the  total  return. 

The  computational  efficiency  of  our  converging  branch  algorithm  is  measured 
with  the  problem  by  changing  the  number  of  discretizations  of  the  variables. 

The  problem  above  was  solved  with  six  discretizations  in  the  variables 
x2i*  and  d2  and  20  discretizations  for  all  other  variables. 


Table  3.8  shows  the  storage  requirement  and  the  computational  time  for 
different  combinations  of  discretizations.  We  see  that  the  computational  storage 
requirement  is  clearly  affected  by  the  number  of  discretizations  of  the  branch 
output  variable  xQ1  as  well  as  the  other  variables. 


Table  3.8  Computational  Storage  Requirement  and  CPU  Time  for  the  Converging 

Branch  System 


Number  of 
discretizations 
of  all  others 

Number  of  discretizations  of  x^^ 

5  10  15  20 

20 

5,712  (.709) 

10,322  (.989) 

14,932  (1,313) 

19,542  (1.674) 

30 

5,852  (1.077) 

10,562  (1.834) 

14,292  (2.484) 

19,982  (3.219) 

40 

5,992  (1,729) 

10,802  (2.913) 

15,612  (4.077) 

20,422  (5.201) 

*  Numbers  in  the  (  )  represent  the  CPU  times  (seconds) 

*  The  storage  requirement  is  measured  with  the  number  of  elements  in  the  arrays, 
actually  used. 

3.6  Computational  Efficiency  of  the  Converging  Branch  Algorithm 

The  computer  program  of  the  converging  branch  algorithm  is  modified  so  that 
any  integer  values  are  permitted  for  input  and  decision  variables  at  each  stage  in  the 
system. 

Assuming  that  the  input  (x^,  x^)  and  decision  variables  (d^  d^)  at  each  stage 
have  their  lower  and  upper  bounds  such  that 


‘ii’ii  ui 

i  =  1, 

2,  ....  N 

Cdi  <d1  <  ud1 

i  =  1, 

2 . N 

Ul  i  *11  i  uil 

i  =  1, 

2,  . . . ,  M 

*dil  i  dil  -  Udll 

i  = 

2 ,  . . .  ,  M 

^01  -  *01  -  uoi 


then  the  computational  storage  requirement  can  be  analyzed  as  follows: 


The  main  storage  requirement  in  the  converging  branch  algorithm  is  for  the 
decision  tables  and  the  return  tables.  Since  the  decision  table  for  the  main 
serial  process  is  constructed  at  each  stage,  the  computational  storage  depends 
only  on  the  number  of  discretizations  of  each  decision  variables.  Hence,  the  decision 
table  for  the  main  serial  process  has  the  following  storage  requirement: 

(Number  of  stages  in  the  main  serial  process)  multiplyed  by  (maximum 
number  of  discretizations  of  the  input  variables  in  the  main  serial 
process)  =  N  X  max  (u^  -  +  1) 

1  <  i  <  S 

The  amount  needed  for  the  branch  system  is  a  little  complex.  Since  at  each 
stage  of  the  branch,  the  transformation  function  has  to  satisfy  .the  constraint 
given  by  the  branch  output  (i.e,  £  x^  <_  u^)  the  storage  requirement  for  the 

decision  table  becomes 

Number  of  stages  in  the  branch)  multiplyed  by  Maximum  number  of 

discretizations  of  the  input  in  the  branch)  multiplied  by  Number 

of  discretizations  of  Xq^)  =  M  X  max  (u^  -  +  1)  X  (u^  -  +  D 

!  <_  i  <  M 

Another  major  source  of  storage  requirement  is  due  to  the  return  table.  Since 
the  system  is  solved  using  a  backward  recursion  equation,  we  need  to  revise  the 
return  at  each  stage  with  respect  to  the  return  at  the  previous  stage.  Hence,  the 
storage  requirement  for  the  return  table  at  the  main  serial  process  becomes 

2  X  {[max  u^]  -  [min  £  +  1]} 

1  <  i  <  N  1  £  i  £  N 

Similarly,  the  computational  storage  of  the  return  table  for  the  branch  system 
is  given  by 

2  X  {[max  u^]  -  [min  £^  +  1J>  X  (uQ1  -  £Q1  +  1) 
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Notice  that  the  storage  requirement  of  the  return  tables  are  largely  dependent 
on  the  size  of  the  input  variables  each  of  which  is  a  function  of  the  decision 
variable  and  input  at  the  previous  stage. 

To  illustrate  the  computational  requirement  of  the  converging  branch  system, 
we  solved  a  set  of  problems  with  different  numbers  of  stages  both  in  the  main 
serial  process  and  in  the  branch.  Table  3.9  shows  the  storage  requirement  and 
computational  time  for  each  case.  In  each  case,  a  set  of  simple  transformations 
and  return  functions  at  each  stage  was  generated.  Five  discretizations  for  the 
branch  output  variable  and  ten  for  other  variables  were  used.  The  lower- 
bounds  of  all  decision  variables  were  assumed  to  be  zero,  and  a  reasonable 
lowerbound  for  each  input  variable  was  given. 

We  can  see  that  the  storage  anu  the  CPU  time  increase  are  very  sensitive  to 
the  number  of  stages  in  the  branch  system. 

Table  3.9 :  Storage  Requirement  and  CPU  Times  for  Different  Number  of  Stages 


N  = Number  of  stages 
in  the  serial  process 

M  = 

Number  of 

Stages 

in  the 

Branch 

2 

3 

4 

5 

470 

(.411) 

620 

(.442) 

770 

(.566) 

10 

620 

(.448) 

770 

(.499) 

920 

(.582) 

15 

770 

(.474) 

920 

(.525) 

1070 

(.605) 

*  Numbers  in  (  )  represent  the  CPU  times  in  seconds. 

*  The  storage  requirement  is  measured  with  the  number  of  elements  in  the  arrays. 
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Chapter  4 


ANALYSIS  OF  FEEDFORWARD  LOOP  SYSTEMS 


4.1  The  Basic  Structure 

A  feedforward  loop  system  is  akin  to  the  diverging  branch  system  in 
which  the  branch  output  feeds  into  the  main  serial  subsystem  at  state  j. 
Thus,  it  may  be  viewed  as  a  simple  combination  of  diverging  and  converging 
systems. .  The  divergence  occurs  at  stage  k  while  the  convergence  takes  place 
at  node  j,  j  <  k.  The  basic  structure  is  diagrammed  in  Fig.  4.1. 


Let  the  transformations  and  returns  which  are  the  same  as  in  the  usual 
serial  ones  for  all  stages  other  than  j  and  k  be  defined  as  follows: 

x  =  t  (x  ,  d  ) 
n-1  n  n  n 


r  =  r  (x  »  d  ) 
n  n  n  n 


,  n  =  1 .  S,  n  )*  j 


Xj-1  “  Cj<X01’  V  v 

rj  =  rj(x01’  xr  V 

xm-l,l  =  Stil^ml’  dml) 

rml  =  rml(xml’  dmL>  ’  m  =  " 


I 

l 

X 

L 

it 

:l  -  - 


*M1  ‘  WV  V 


Vl  *k 


I  xii 

Ml  - 11 


Figure  4.1:  A  Feedforward  Loop  System 


Clearly,  this  is  a  more  difficult  system  to  treat  than  any  of  the 
systems  discussed  so  far.  A  fundamental  observation  which  supports  this 
viewpoint  is  the  fact  that  the  branch  is  both  diverging  and  converging. 

Thus,  in  nature  the  branch  input  as  well  as  the  output  affects 

the  return  from  the  serial  system.  As  a  consequence,  of  this  important 
fact,  if  the  branch  is  optimized  separately  as  a  serial  system,  its 
optimal  return  must  be  determined  as  a  function  of  both  its  input  and  out¬ 
put.  In  effect,  we  have  a  two  point  boundary  value  problem.  There  are 
thus  at  least  two  possible  routes  to  the  computational  scheme.  The  optimal 
branch  return  can  be  absorbed  into  the  main  serial  system  either  at  the 
converging  stage  j  or  the  diverging  stage  k.  In  either  case,  it  must  be 
noted  that  a  two  state  variable  dynamic  programming  problem  results  for 
the  branch  optimization.  To  minimize  the  computational  burden,  adroit 
schema  must  be  sought  to  reduce  the  vector  optimization  problems  to  that 
of  a  series  of  one  variable  optimization  problems. 

4 . 2  The  Optimization  Procedure 

Here,  we  will  give  a  procedure  which  solves  the  feedforward  loop 
system  in  which  the  absorption  of  the  branch  occurs  at  stage  k.  Nemhauser  [22] 
describes  both  approaches  but  indicates  why  the  absorption  at  the  diverging 
stage  is  the  preferred  procedure.  In  this  procedure,  there  are  essentially 
two  main  steps  involving  the  loop  system  and  the  main  serial  system. 

4.2.1)  Optimization  of  the  Loop  System  (from  stage  11  to  Ml) 

The  branch  consisting  of  stages  11  through  Ml  is  optimized  to  find 
f Mi ( »  xoi>*  This  procedure  is  the  same  as  the  one  in  the  converging 
branch  system  treated  earlier.  Notice  that  this  is  a  two  state  variable 
dynamic  program. 
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4.2.2)  Optimization  of  the  Main  Serial  System 

The  following  five  phases  in  the  procedure  may  be  considered. 


4.2.2-i)  For  the  stages  from  1  to  j  -  1  (from  node  1  to  node  preceding 
the  convergent  node  j) 

The  optimal  return  from  stage  1  through  j  -  1  is  obtained 
by  using  the  usual  recursive  procedure,  i.e.. 


fl(Xi)  =  max  r  (x  ,  d^) 
dL 

f  (x  )  =  max  [ r  (x  ,  d  )  +  f  . (t  (x  ,  d ))],  n  =  2,...,  j-1 
n  n  ,  n  n  n  n-1  n  n  n 
a 

n 

4.2.2- ii)  For  stages  from  j  to  k  -  1  (from  convergent  node  to  node 

proceeding  divergent  node  k) 

Since  the  absorption  of  the  loop  is  assumed  to  occur  at 
stage  k,  the  optimization  of  xQ^  is  deferred  to  stage  k. 

Hence,  x^  is  carried  as  a  state  variable  in  stages  j 
through  k  -  1.  Let  the  optimal  return  be  defined  as 

Vh-  *oi>- 

The  resultant  recursive  equations  are 

Vv  V  =  [rj°y  xoi’  V  +  fj-i(tj(v  *oi’  V)] 

j 

and  f(x,  x_  )  =  max  [r  (x  ,  d  )  +  f  (t  (x  ,  d  ),  x  )] 
nnUl  ^  nnn  n— 1  n  n  n  Ul 

n 

n  )  •  •  •  y 

4.2.2- iii)  Optimization  of  Xq^  at  stage  k  (absorption  of  divergent  node) 

At  stage  k,  the  optimal  loop  return  x^)  is  a‘,sor^,ed 

into  the  main  serial  process  and  we  have  the  following 
recursion  equation: 

W<xk>  ■  max.(ck<V  V  +  WW  V-  xoi’ 

X01*  dk 

+  fMl(tkl(xk’  dk)’  x01^ 
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At  this  stage,  the  optimal  branch  output  Xq^(x^)  is 
obtained  as  a  function  of  the  input  variable  x^»  of  the 
stage  k. 

4.2.2-iv)  For  the  stages  from  k  +  1  through  N  (remainder  of  the  serial 
system) 

The  recursion  equation  for  stages  k  +  1  through  N  is  given 
as  usual  by 

f  , (x  )  =  max  [r  (x  ,  d  )  +  f  (t  (x  ,  d  ))], 

n+Ml  n  ,  n  n  n  n-l+Ml  n  n  n 
a 

n  n  =  k+1 ,  ....  N 

This  concludes  the  optimization  phases  for  the  situation  in  which 
absorption  takes  place  at  the  divergent  node.  When  it  takes  place  at  the 
convergent  node  j,  a  different  set  of  recursion  equations  is  needed.  In 
general,  the  same  number  of  computations  is  required  in  either  case.  The 
main  difference  in  the  procedure  results  from  the  consideration  of  where 
is  optimized.  In  the  convergent  node  absorption  variety,  the  optimization  is 
done  earlier  in  the  analysis  with  x^  being  carried  as  a  state  variable  from 
stages  j  +  1  to  k  -  1  whereas  x^  is  carried  as  a  state  variable  during 
similar  periods  when  the  optimization  is  done  at  the  divergent  node  k  as 
just  described. 


I 
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Figure  4.2:  Flow  Chart  for  the  Feedforward  Loop  System 


Read  the  input  data 


K  =  it  of  stages  in  the  aain  serial  process 
H  =  It  of  stages  in  the  loop 

the  converging  stage  of  the  loop 
and  the  diverging  stage  of  the  loop 
Kn(K  =  discretization  It  of  input  at  stage  n(ml) 

Pn(Pml)  =  discretization  It  of  decision  at  stage  n(ml) 

BK  (BK  )  =  lower  bound  of  input  at  state  n(ml) 

BP  (BP  ,)  =  lower  bound  of  decision  at  stage  n(ml) 

n  ml 

I  ~ — 

Define  the  transition  and  return  functions 

Si’  Si’  tkl’  tnl*  Sal 


for  n  =  1,  2,  N  and  tn  **  1,  2,  M 
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The  flow  chart  for  the  numerical  solution  of  the  feedforward  loop 
system  is  shown  in  Figure  4.2.  To  illustrate  the  algorithm,  a  computer 
program  was  written  and  implemented  using  the  test  problem  described  in 
Figure  4.3.  In  the  example,  it  is  desired  to  maximize  the  sum  of  the 
stage  returns  for  the  problem  with  the  structure  given  in  Figure  4.3  using 
the  data  in  Table  4.1.  In  this  example,  the  convergent  stage  is  node  2  and 
the  divergent  stage  is  node  5.  The  loop  contains  nodes  31,  21,  and  11. 


x„  x 

i  2  i 


Figure  4.3:  A  Feedforward  Loop  System  Example 


Table  4.1:  The  Return  and  Transition  Function  for  the  System  of  Fig.  4.3 


Stage  Decision  Return 


Transition  Constraints 


(xQ1  +  x2)/2  +  d2 

*11  +  dll  0  5  Xi  ^  15,  11,1 . 4 

*21  +  d21  °  *  XilS  10,  l“0,  lf  2*  3 

*31  +  d31  0  *  X5  *  3 


x  +  d 
x3  3 

x.  +  d. 


0  <  dt  S  2,  i-1,..., 

0  *  d,,S  2,  i-1,  2,  3 


5 


Our  problem  can  be  formulated  using  the  following  mathematical  form: 

5  2 

max  1  r  +  £  r 

.  n  ,  ml 

n=l  m=l 

s.t.  0  £  x  £15  n=l,  . 4 

n 

0  £  x5  £  3 

0  S  Xml  S  10  m  =  0,  1,  ....  3 

0  £  d  £2  n  =  1 . 5 

n 

°  S  d^  £  2  m  =  1,  2,  3 

The  computational  results  are  shown  in  the  computer  output  of 
Table  4.2  The  optimal  decision  and  return  at  each  stage  are  summarized  in 
Table  4. 3 . 
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Table  4.2:  Computer  Output  of  Feedforward  Loop  Example 

in  Figure  4.3 


DECISION  TABLE  AT  STAGE  11 


0,  1 .  2 ********************************************************** 
**  0.  1.  2  ♦  **************************************************** 
*******  0,  1 »  2 » *********************************************** 
************  0.  1.  2 » ****************************************** 
*****************  0.  1.  2 ♦ ************************************* 
**********************  0.  1*  2 . ******************************** 
***************************  0.  1*  2 » *************************** 
********************************  0.  1.  2 ♦ ********************** 


*************************************  0.  1. 
******************************************  0*  i , 

***********************************************  0 , 


2 ♦ ***************** 
2 ♦ ************ 


2* ******* 


*;?* 
**  k 
**' 

■*>  if  ‘l- 

v><  \b 

^  ip  ^ 
*** 
*** 
*** 


DECISION 

TABLE  AT 

STAGE 

21 

, 

0, 

1* 

2. 

2. 

2. 

2, 

2. 

2. 

2. 

2. 

n 

2. 

2 . 

2. 

0. 

1. 

2, 

2. 

2. 

2. 

2. 

2  ♦ 

2. 

2* 

n 

2. 

^7 

A.  * 

2 

2. 

2. 

0. 

1 . 

2. 

2. 

2. 

2. 

2. 

2. 

2. 

2. 

2. 

2. 

2 

2. 

2. 

2. 

0. 

1, 

2. 

2. 

2. 

2* 

2, 

2. 

2. 

2. 

2. 

2 

2. 

2. 

2. 

2. 

0. 

1, 

2. 

2. 

2. 

2. 

2. 

‘  2. 

2. 

2. 

2 

2. 

2. 

2. 

2. 

2. 

0. 

1 . 

2. 

2. 

2. 

2* 

2. 

2. 

2* 

2 

2, 

2. 

2. 

2. 

2. 

2. 

0. 

1. 

2. 

2. 

2. 

2. 

2, 

2. 

2 

2. 

2. 

2. 

2. 

2, 

2. 

2. 

0. 

1. 

2. 

2. 

2. 

2. 

2, 

2 

2. 

2. 

2. 

2. 

2, 

2, 

2. 

2. 

0. 

1. 

2. 

2. 

2. 

2. 

2 

2, 

2. 

2. 

2. 

2. 

2, 

2. 

2. 

2. 

0. 

1. 

0. 

1. 

2. 

2 

2, 

2. 

2, 

2. 

2. 

2. 

2. 

2. 

2. 

2. 

0. 

0, 

0, 

2. 

2 

DECISION 

0.  1. 

TABLE 

2. 

AT  STAGE 

2. 

31 

2. 

2. 

2, 

2, 

2* 

2, 

2, 

2, 

2. 

2, 

0. 

1. 

2» 

2. 

2. 

2 « 

2, 

2. 

2. 

2. 

2. 

2. 

2. 

2 

2 » 

2. 

0. 

1. 

2. 

2. 

2. 

2. 

2, 

2. 

2. 

2. 

2. 

2. 

o 

2. 

2, 

2. 

0. 

1. 

2. 

2. 

2. 

2, 

2. 

2. 

2. 

2. 

2* 

2 

2. 

2. 

2* 

2. 

0. 

1. 

2. 

2. 

2. 

2. 

2. 

2  ► 

2. 

2* 

2 

2. 

2. 

2. 

2. 

2. 

0. 

1. 

2. 

2. 

2. 

2. 

2. 

2. 

2. 

2 

2. 

2. 

2. 

2. 

2, 

2. 

0. 

1. 

2. 

2. 

2, 

2. 

2. 

2. 

2 

2. 

2. 

2. 

2* 

2. 

2. 

2. 

0. 

1. 

2. 

2, 

2. 

2. 

2* 

2 

2, 

2. 

2, 

2. 

2. 

2. 

2. 

2. 

0. 

1. 

2, 

2, 

2. 

2. 

2 

2. 

2, 

2. 

2. 

2* 

2. 

2, 

2. 

2. 

0, 

1 . 

1. 

1. 

2. 

2 

2. 

2. 

2 . 

2. 

2, 

2  * 

2. 

2. 

2 « 

2. 

0. 

0. 

0. 

2. 

2 

67 


Table  4.2  Continued 


I 

I 

I 

I 

] 


T 


■l 

*T 


t 

l‘ 

- 

S 

i 

I1 

|j. 

i 

f 

i 

f 

f 

f 

I 


2. 

2.  2. 

n 

» 

0. 

0,  0, 

0. 

0. 

0.  0. 

0. 

0. 

0.  0. 

0. 

0. 

0.  0. 

0. 

0. 

0.  0. 

0. 

0. 

0.  0. 

0. 

0. 

0,  0. 

0. 

0. 

0.  0. 

0. 

0. 

0.  0, 

0. 

0. 

0.  0 . 

0. 

0. 

0.  0. 

0. 

0. 

0,  0. 

0. 

0. 

0.  0. 

0. 

0. 

0.  0. 

0. 

* OPTIMAL  LOOP 

OUTPUT  FOR 

EACI 

*QF  THE  DIVERGING  STAGE  h 

0 

8 

1 

9  v 

9 

10 

3 

11 

•STAG 

E  INPUT 

DEC IS  RETURN 

5 

3 

2 

5 

4 

5 

2 

7 

3 

7 

7 

9 

2 

9 

o 

99 

1 

12 

9 

14 

31 

5 

2 

9 

21 

7 

2 

11 

11 

9 

2 

13 

TOTAL 

OPTIMAL  RETURN  IS 

90 

EOI,  0  FILES.  1  REGS,  679  WORDS » 


/ 
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Table  4.3  :  Optimal  Input.  Decision,  and  Return  of  the  System  In  Figure  4. 


Stage 

Opt.  Input 

Opt.  Decision 

Opt .  Return 

5 

3 

2 

5 

4 

5 

2 

7 

3 

7 

2 

9 

31 

5 

2 

9 

21 

7 

2 

11 

11 

9 

2 

13 

2 

9  (11)* 

2 

22 

1 

12 

2 

14 

3 


*  (  )  represents  the  input  from  the  feedforward  loop 


4.4  Analysis  of  the  Feedforward  Loop  System 

The  storage  requirement  of  the  feedforward  loop  system  can  be 
characterized  by  considering  the  following  two  aspects: 

1.  The  storage  requirement  for  the  loop  system  can  be  analyzed  as 
the  converging  branch  system. 

2.  The  storage  requirement  for  the  main  serial  system  is  much  higher 
than  that  for  the  diverging  or  converging  branch  system.  This 

is  because  Xq^  is  carried  as  a  state  variable  for  stages  j  through 
k  -  1. 

Now,  let  U^,  2.^,  and  2,  ^  be  defined  as  follows: 

S  Xf  S  i  ■  1,  2,  . ..,  N 

lil  ^  Xil  5  Uil  i  -  0,  1,  ...,  M 

Also,  let  and  K  be  respectively  the  number  of  discretization  levels 
for  variables  x^  and  x^,  and  K  and  K^be  defined  as  follows: 
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K  =  max  K . 

1 

1  <  i  <  N 

K,  =  max  K., 

1  ll 

0  s  i  <  M 

Since  the  loop  optimization  must  be  treated  as  an  initial-final  value 
problem  as  in  the  converging  branch  system,  the  storage  requirement  for  the 
loop  system  becomes 

SRL  =  MK  K  +  2  K  (max  U  -  min  5..,) 

1  01  01  .  ll  .  ll 

1  1 

The  first  term  of  the  above  equation  is  the  storage  requirement  for 

the  decision  value  for  each  pair  of  input  and  branch  output  values.  The 
second  term  is  the  storage  requirement  for  the  stage  returns  for  each 
possible  pair  of  input  variable  x  and  branch  output  x., . 

Now,  the  analysis  of  the  main  serial  system  of  the  loop  structure  is 
different  from  the  one  for  the  converging  branch  system.  In  the  converging 
branch  system, the  branch  return  is  combined  at  the  converging  stage  S  with 
the  returns  from  main  serial  process  and  then  the  branch  output  is 
optimized  as  a  function  of  xQ.  However,  in  the  loop  structure,  x  cannot 
be  optimized  at  the  converging  stage  j,  since  all  the  recursive  returns 
from  stage  j  to  k  -  1  are  affected  by  the  variable  x^.  Hence,  xQ1  needs 
to  be  carried  as  a  state  variable  for  stages  j  through  k  -  1.  Thus,  the 
storage  requirement  for  the  main  serial  system  becomes 

SRiMS  =*  NKKnl  +  2  Kn,  (max  U.  -  min  £.) 

01  01  ^  i  i 

Thus,  the  total  storage  requirement  for  the  loop  structure  becomes 

TRSL  =  MK, K„.  +  2  K„. (max  U,  -  min  fc.,} 

1  01  0J  j,  i  ll 

+  NKKQ1  +  2  KQ1  (max  U.  -  min  l  ) 
i  1  i 
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If  we  assume  K,  =  K,  max  U.,  =  max  U .  and  min  8,.,  =  min  5.  ,  then  the  above 
1  ll  i  ll  i 

requirement  becomes 

TRSL  =  (M  +  N)  KKQ1  +  2  KQ1  (max  U±  =  min  i.  ) 

4.5  Computational  Experiments  with  the  Loop  System 

In  Table  4.4  we  will  illustrate  the  computational  requirement  of  the 
feedforward  loop  system.  We  solved  a  set  of  problems  with  different  numbers 
of  stages  in  the  loop.  For  each  problem  the  following  input  data  were  used. 

N  =  5 

K  =  K  =  15 

K01  =  10 
max  U .  =  14 

i 

min  £ .  =  0 
i 

As  shown  in  the  table,  both  the  storage  requi~ement  and  the  CPU  time  can  be 
represented  as  a  linear  function  of  the  number  of  stages  in  the  loop. 

Table  4.4:  Computer  Storage  Requirements  for  the  Feedforward  Loop  System 


Number  of  Stages 
in  the  Loop 


Storage  Requirements 


CPU  Time 
(Seconds) 
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ANALYSIS  OF  FEEDBACK  LOOP  SYSTEMS 
5. 1  The  Basic  Structure  and  Algorithm 

A  feedback  loop  system  is  identical  to  a  feedforward  system  except 
that  the  relative  positions  of  stages  j  and  k  are  reversed.  In  the  feedback 
loop  system  as  shown  in  Figure  5.1,  k  <  j,  where  k  is  the  diverging  stage 
and  j  the  converging  stage. 

The  transition  and  return  functions  for  the  feedback  loop  are  identical 
to  those  of  the  feedforward  loop  with  the  numbering  of  the  stages  as  shown 
in  Figure  5.1. 

The  recursion  equations  for  the  feedback  loops  are  a  little  different 
because  of  the  different  positions  of  stages  j  and  k.  The  optimization 
procedures  for  the  stages  in  the  loop  as  well  as  those  in  the  main  serial 
process  except  stages  j  and  k  are  the  same  as  in  the  feedforward  loop 
systems.  Hence,  we  will  not  give  the  detailed  optimization  procedures  for  the 
feedback  loop  systems.  Instead,  we  will  summarize  the  'lgorithm  in  the  flow 
chart  given  in  Figure  5.2. 

In  the  algorithm  described  in  the  flow  chart,  the  optimal  loop  return 
^Ml^XMl’  XQ1^  combi-ne<*  with  the  main  serial  system  at  the  diverging 
stage  k,  and  stages  k  +  1  through  j  -  1  are  optimized  to  give  f xoi^ ' 
The  branch  output  is  then  optimized  at  the  converging  stage  j. 

The  analysis  of  the  feedback  loop  system  is  not  given  since  the  com¬ 
plexity  of  this  system  is  identical  to  that  of  the  feedforward  loop  structure 
presented  in  section  4.4. 
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Figure  5.2:  Flow  Chart  for  the  Feedback  Loop  System 


ml  ml 


Read  the  input  data 

N  =  ir  of  stages  in  the  main  serial  process 

M  ■  if  of  stages  in  the  loop 

=  converging  stage  of  the  loop 

=  diverging  stage  of  the  loop 

K  (1C  . )  =  discretization  If  of  input  at  stage  n(ml) 
n  mi. 

P  (P  ,)  =  discretization  //  of  decision  at  stage  n(ml) 
n  ml 

BKn(BKml)  =  lower  bound  of  input  at  stage  n(ml) 
BPn(BPml)  =  lower  bound  of  decision  at  stage  n(ml) 


Define  the  transition  and  return  functions 
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ml 
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To  illustrate  the  optimization  of  a  feedback  loop  system,  we  consider 
an  example  with  the  structure  shown  in  Figure  5.3  and  the  data  given  in  Table  5.1  . 


fl 


The  objective  is  to  maximize  the  sum  of  the  stage  returns. 


31 

X21 

21  [ 

-  J 

Figure  5.3:  A  Feedback  Loop  System 


Table  5-1:  The  Return  and  Transition  Functions  of  the  System 

in  Figure  5.3 
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1 


la  Table  5.3  we  display  a  summary  of  the  results  of  this  example  problem 
while  the  decision  table  at  each  stage  is  shown  in  the  computer  output  of  Table 


5.2. 


Table  5.3:  Optimal  Input,  Decision,  and  Return  from  Table  5.2 


Stage 

Opt.  Input 

Opt.  Decision 

Opt , 

.  Return 

31 
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10 

21 

8 
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12 

11 

10 
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14 
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3 

7 

5 

7  (12)* 
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22 

« 

4 

11 
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14 
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14 
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17 
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17 
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20 
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20 
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22 

*  (  )  represents 

the  optimal  input 

from  feedback  loop  to 

stage  5 

j 
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Chapter  6 


SOLUTION  OF  SPECIALLY  STRUCTURED  NONSERIAL  NETWORKS  VIA 
THE  IMBEDDED  STATE  SPACE  DYNAMIC  PROGRAMMING 

6.1  Introduction 

In  this  chapter,  we  want  to  show  how  certain  nonserial  networks  with 
special  structures  can  be  solved  using  some  recently  developed  dynamic  programming 
algorithms.  The  structure  in  question  refers  not  so  much  to  the  network  configura¬ 
tion  as  it  does  to  the  nature  of  the  return  functions  and  the  constraint  spaces. 

For  example,  when  the  return  functions  are  discontinuous  and,  in  particular,  are 
(transformable  to)  step  functions,  the  imbedded  state  space  dynamic  programming 
developed  by  Esogbue  and  Morin  [20]  is  an  especially  potent  procedure.  This 
method  mitigates  the  usual  curse  of  dimensionality  problem  inherent  in  most 
dynamic  programming  solutions  by  reducing  an  M-dimensional  search  problem  to  a  one 
dimensional  dynamic  programming  over  a  sequence  of  imbedded  state  spaces. 

An  excellent  application  of  this  procedure  to  the  multidimensional  knapsack 
problem  was  discussed  by  Morin  and  Esogbue  while  Morin  and  Marsten  [21]  give  the 
details  of  an  algorithm  developed  for  the  computational  solution  of  large  scale 
dynamic  programming  problems .  • 

6.2  Outline  of  the  Imbedded  State  Space  Approach  (ISSA) 

The  key  to  this  approach  is  the  exploitation  of  the  discontinuity  preserving 

properties  of  the  maximal  convolution  to  transform  a  search  over  the  entire  state 

space  to  one  restricted  to  a  set  of  imbedded  state  spaces.  This  proposition  is 

propounded  in  [20]*  We  outline  the  procedure  in  the  sequel. 

Consider  the  following  multidimensional  knapsack  problem  in  which  the 

r  (*)  are  discontinuous  and  furthermore  step  functions: 
n 

N 


£  r ,  ( x  ) 
j  =  l  J  J 

(1) 

N 

jl,  8ij'V 1  bi  •  1  ■ 11 2 . H 

(2) 

xj  '  Sj  .  j  ■  1.  2 . N 

(3) 

oc 


where  =  {0,  1,  2,  K  ^ }  and  -*■  R+  is  nondecreasing  with  r^(0) 

=  0,  V.  .  ;  g..  :  S  -*•  R  with  g..(0)  =  0  and  b  =  (b  ,  b.,  b  }  7/,  0. 

l  j  J  j  '  i  j  d.  z  m 

Let  f(n,  S)  be  the  maximum  objective  function  value  of  an  undominated 

feasible  solution  to  (1),  (2),  and  (3)  in  which  only  the  first  n  variables 

(x.. ,  x„ ,  . ..,  x  )  can  be  positive  and  whose  resource  consumption  cannot  exceed 
1  1  n  N  N  N 

8  =  (B, ,  80,  •••.  8  )  •  When  E  r.  (x.)  <  I  r.(x.)  and  Z  g..(x.)  > 

12  n  j=l  J  J  —  j=i  J  1  j=1  ij  3  “ 

N 

E  g..(x.)  with  strict  inequality  holding  in  at  least  one  of  the  (M+l)  inequalities 
j=l  13  3 

the  feasible  solution,  x  =  (x, ,  x„ ,  ....  x  )  is  said  to  be  dominated  by  the  feasible 

1  i  n 

solution,  x  =  (x, ,  X. ,  . . . ,  x  ) . 

1  l  n 

For  0  <  n  <  W,  let  R  be  the  (domain)  set  of  resource  consumption  vectors 
—  —  n 

g  (k)  =  [g  (k) ,  g_  (k) ,  ....  g„  (k) ]  of  all  undominated  feasible  values  of  x  =  k. 
n  J  ti  Zn  Zci  n 

Also,  for  1  <_  n  <_  N,  let  F^  be  the  set  of  resource  consumption  vectors  B  of  all 

undominated  feasible  solutions  (x. ,  x_,  ....  x  )  to  the  following  subproblem: 

l  4  n 


max  Z  r . (x . ) 
j=l  3  3 


c.  ^  SyOtj,  i  b1 

‘j  '  S1 


(4) 

=  1,  2,  . . .  ,  M 

(5) 

=  1,  2,  . . .  ,  N 

(6) 

Then,  as  demonstrated  in  [  ] 

F  r  {R  F  i  u  <R  ©  F„  >  n  =  1,  2,  ....  N  (7) 

n  ^  n  n-1  ^  n  n-1 

where  (R  ©  F  ,)*  denotes  the  set  obtained  by  forming  all  sums  of  one  element  of 
n  n-1 

R  with  one  element  of  F  The  algorithm  proceeds  by  recursively  generating,  for 

n  n-1 

all  1  <  n  <  N,  all  feasible  candidates  for  F  from  F  ,  and  R  via  the  following 
—  —  n  n-i  n 

functional  equation: 

f (n,8)  =  (r  (k)  +  f(n-l,  B-g  (k)  |  g  (k)  e  R  ,  (8) 

n  n 

( B— g  (k) ]  e  F  ,  6  <  b} 
n  n-i  — 
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and  the  boundary  condition 


f(0,0)  =  0  (9) 

The  implication  of  the  foregoing  is  that  instead  of  calculating  f(n,  B) » 

V  3  e  ft,  we  only  need  to  calculate  it  for  3  €  F  while  noting  that  f  can  be 

n  n 

constructed  recursively  from  and  Fn_^*  Furthermore,  we  can  usually  eliminate 

certain  elements  of  either  R  u  F  ,  or  R  @  F  ,  as  being  inefficient  or  infeasible 

n  n-1  n  n-1 

thereby  reducing  the  list  length  of  F 
'  n. 

In  this  way,  an  M-dimensional  search  problem  is  reduced  to  a  one  dimensional 
dynamic  programming  problem  on  the  sequence  of  imbedded  state  spaces  F^,  F^,  ...» 

C  ft*  An  algorithm  using  the  above  procedure  to  construct  the  successive  imbedded 
state  spaces  and  terminate  with  F^  is  illustrated  in  Table  6.1. 

6.3  An  Application  of  ISSA  to  a  Feedforward  Loop  System 

Consider  the  following  nonserial  network  where  each  stage  has  a  return  expressed 
as  a  function  of  the  input  variable  and  each  of  the  input  variables  has  some  con¬ 
straints  that  need  to  be  met.  This  is  a  simple  feedforward  loop  system  with  con¬ 
straints. 


Figure  6.1:  An  Example  of  a  Nonserial  Network 

(Feedforward  Loop) 

Suppose  we  have  one  constraint  for  the  main  serial  process  and  the  other  for  the 
feedforward  branch.  That  is,  the  first  is  for  stages  1,  2,  and  3  and  the  second 
for  stages  1,  4,  5,  and  3.  Then,  our  problem  can  be  formulated  as  follows:  (10)  -  (13) 

i 


Table  6.1:  Seeps  of  the  Imbedded  State  Space  Algorithm  for  the  Multidimensional 

Knapsack  Problem 

This  algorithm  may  be  decomposed  into  the  following  steps: 

Step  1  Set  n=0,  Fq  =  (6°)  and  f(o,0°)  -  0,  where  6^  =  0 

Step  2  n  ^  n+1  and  - 0 

Step  3  If  n  >  N,  stop. 

Step  4  V  =  {0°,  g1,  ....  6P}  ,  where  P  =  |  F  x  |-  1 

Step  5  F  . _ F  . 

n  "4 —  n-1 

Step  6  k  < —  k+l  If  k  >  K,  go  to  step  2. 

Step  7  gn(k)  =  ^8ln^k)’  g2n(k^’  ***»  8^00] 

Step  8  p  0 

Step  9  If  gn00  +  SP  is  inf easible  go  to  step  13. 

Step  10  If  g^Ck)  +  3P  is  dominated  by  some  point  in  F^,  go  to 

Step  13. 

Step  11  F4 - F  U  {g  (k)  +  SP> 

n  n  n 

f (n,  (g  (k)  +  gP))< — r  (k)  +  f (n-1,  gP) 
n  ^  n 

Step  12  Eliminate  all  the  dominated  points  from  Fr.  i.e.,  F^  ^ _ Fr  minus 

{all  points  dominated  by  gR(k)  +  8P) 

Step  13  p  4 —  P+1 

Step  14  If  P  £  P»  g°  t°  step  9. 

Otherwise,  go  to  step  6. 
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(10) 


max  r]/xi^  +  r2^x2^  +  r3^X3^  +  r4^X4^  +  r5^X5^ 


11<V 

+  g12*x2^  +  Si3(x3)  -  bl 

(11) 

21<xl> 

+  823^X3^  +  824^X4^  +  S25^X5^  —  b2 

(12) 

x.  e  S . 

(13) 

where  r^(x^)  and  8ij(xj)  represent  the  return  and  elements  of  the  i-th  constraint 
at  stage  j  respectively. 


Let  b  =  (6,  10)  and  the  values  of  the  r^  and  functions  be  tabulated  as  in 
Table  6.2. 


Table  6.2:  Input  Data  for  the  Example 


H 

n 

r2 

812 

g22 

r  3 

S13 

823 

H 

0 

0 

0 

0 

0 

0 

0 

m 

0 

0 

0 

0 

0 

0 

0 

i 

2 

1 

2 

4 

2 

4 

3 

4 

4 

0 

3 

3 

0 

8 

2 

4 

2 

4 

5 

3 

0 

6 

6 

8 

5 

0 

5 

5 

0 

11 

3 

5 

3 

6 

9 

4 

0 

7 

7 

12 

8 

0 

6 

6 

0 

15 

Notice  the  special  structure  of  this  problem  in  which  the  RHS  are  constants  as 
opposed  to  the  usual  system  (see  example  4.3)  in  which  they  are  expressed  as 


functions  of  the  input  state  and  decision  variables. 

We  will  now  illustrate  how  the  imbedded  state  space  algorithm  whose  steps 
were  given  in  Table  6.1  may  be  used  to  solve  the  above  nonserial  network  problem. 

Consider  the  five  stage  problem  as  follows: 

For  Stage  1:  \'x  -  ((0,  0),  (1,  2),  (2,  4),  (3,  6)};  FQ  -  {(0,  0)};  Vx  ©  FQ  - 

{(0,  0),  (1,  2),  (2,  4),  (3,  6)}.  Since  no  points  are  infeasible  or  dominated  we 
have  Fa  -  {(0,  0),  (1,  2),  (2,  4),  (3,  6)}.  The  following  tables  are  then  the 
result  of  the  computations  at  the  first  stage: 


FLIST  PERMUTE  TRACE 


0 

0 

0 

2 

2 

2 

1 

0 

0 

0 

1 

2 

2 

3 

3 

3 

2 

1 

1 

1 

2 

4 

4 

4 

4 

4 

3 

1 

2 

1 

3 

6 

5 

0 

0 

0 

4 

1 

3 

1 

(M+l)  (M+2) 


The  fourth  column  appended  to  the  PERMUTE  table  points  to  the  corresponding  entry 
in  the  TRACE  table 

For  Stage  2:  V2  =  {(0,  0),  (2,  0),  (3,  0),  (4,  0)};  V2  ©  Fx  =  {(0,  0),  (1,  2), 

(2,  4),  (3,  6),  (2.  0),  (3,  2),  (4,  4),  (5,  6),  (3,  0),  (4,  2),  (5,  4),  (6,  6), 

(4,  0),  (5,  2),  (6,  4),  (7,  6)}.  The  point  (7,6)  is  infeasible  since  *  (6,  10). 

The  point  (2,  0)  dominates  the  point  (2,  4).  In  the  same  way,  (3,  0)  dominates 

♦ 

(3,  6)and  (3,  2),  (4,  0)  dominates  (4,  4)  and  (4,  2),  (5,  2)  dominates  (5,  6)  and 
(5,  4).  Finally,  the  point  (6,  4)  dominates  (6,  6).  So  we  have  F2  -  {(0,  0),  (1,  2), 
(2,  0),  (3,  0),  (4,  0),  (5,  2),  (6,  4)}  with  the  following  results: 


FLIST  PERMUTE  TRACE 


For  Stage  3:  V3  -  {(0,  0),  (3,  4),  (6,  8),  (9,  12)},  V3 
points,  of  which  16  are  infeasible  and  5  are  dominated. 
{(0,  0),  <1,  2),  (2,  0),  (3,  0),  <4,  0),  (5,  2),  (6,  4)} 


®  F 2  contains  28 
This  leaves  F3  *  = 

with  the  same  tables 


given  in  Stage  2. 

For  Stage  4:  V4  {(0,  0) ,  (0,  3),  (0,  5),  (0,  6)},  V4  @  F3  contains  28 
points  all  of  which  are  feasible.  Two  points  are  dominated.  Finally,  we  have 
Fa  =  {(0,  0),  (1,  2),  (2,  0),  (3,  0),  (4,  0),  (5,  2),  (0,  3),  (1,  5),  (2,  3), 

(3,  3),  (4,  3),  (5,  3).  (0,  5),  (1,  7),  (2,  3),  (3,  5),  (4,  5),  (5,  7),  (0,  6), 

(1,  8),  (2,  6),  (3,  6),  (4,  6),  (5,  8),  (6,  10)}.  At  the  end  of  this  stage,  we 
have  the  following  tables  for  the  FLIST,  PERMUTE,  and  TRACE  computations  (see  next 
page) . 

We  have  used  X's  in  the  tables  to  indicate  spaces  which,  although  not  currently 
filled,  are  useable.  Although  these  appear  only  in  the  PERMUTE  tables  both  for 
stages  2  and  4  in  our  computation,  they  should  be  used  in  either  the  FLIST  or 
PERMUTE  tables  at  any  stage  where  their  use  in  dictated. 

Notice  that  the  tables  begin  to  look  alike  as  the  computational  process  proceeds 


to  the  final  stage  computation.  This  is  as  it  should  be.  Let  us  now  complete  the 
computations  for  the  stage  5,  the  final  stage. 


At  Stage  5:  V5  -  {(0,  0),  (0,  8),  (0,  11),  (0,  14)}.  V5  ©  F4  contains  100 
points  of  which  69  are  infeasible  and  6  are  dominated.  As  a  result,  we  have 
and  the  resulting  tables  are  the  same  as  in  stage  4. 

Now  we  need  to  find  the  optimal  solution  at  each  stage.  The  zero  in  the 

third  column  of  the  PERMUTE  table  shows  that  the  maximum  return  is  achieved  at 

* 

b  =  (6,  10)  with  objective  function  value  21.  To  reconstruct  the  optimal  x  ,  we 

* 

go  to  the  TRACE  entry  in  row  28.  Here,  we  find  that  x^  *»  3.  Proceeding  to  row  9 

*  * 

of  the  same  table,  we  find  x2  =  3.  Finally,  in  row  3  we  have  Xj,  -  2.  All  other 
variables  are  zero  and  we  have  x  «  (2,  3,  0,  2,  0)  as  the  optimal  solution  and 
optimal  value  of  21. 


6.4  Analysis  of  the  Imbedded  State  Space  Technique 

We  will  now  analyze  the  storage  requirement  of  the  successive  imbedded 
state  space  technique  in  solving  the  following  nonlinear  knapsack  problems. 


N 

max  z  r  (x  ) 

j-1  J  J 

S‘t’  E  8ij(xj)  -  bl  »  i  =  1,  ...»  M 

Xj  e  Sj  ,  j  =  1,  ...»  N 


where  =  {0,  1,  2,  ...»  k^  }  for  all  j. 

To  simplify  our  analysis  we  will  assume  that  the  maximum  permissible  value 
for  each  variable  is  taken  to  be  the  same,  i.e.  k^  “  K  for  all  j.  Then  the 
storage  requirement  of  the  technique  can  be  analyzed  by  considering  the  following 
three  aspects: 

6.4.1)  Storage  Requirements  for  the  Input  Data. 

The  algorithm  needs  r^CXj),  8ij(Xj)  and  bA  as  the  input  data.  Since  we 
have  M  constraints,  N  variables  and  K  discretization  levels  of  each  variable, 
the  storage  requirement  for  r^(x^)  ®nd  ®ij^xj^  becomes  (M  +  1)  KN.  To  store 

the  available  resources  the  algorithm  requires  M  elements  in  an  array.  Hence  the 

storage  requirement  for  the  input  data  becomes  (M  +  1)  KN  +  M. 
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6.4.2)  Storage  Requirement  for  the  Three  Main  Tables. 

The  three  main  tables  in  the  imbedded  state  space  algorithm  are  the 

FUST,  PERMUTE,  and  TRACE  tables.  FLIST  table  shows  the  resource  used  and 

return  for  each  undominated  feasible  solution.  PERMUTE  table  gives  the  best, 

second  best,  ...  solutions  at  each  stage  in  conjunction  with  the  FLIST  table. 

Finally,  the  TRACE  table  is  used  in  generating  optimal  solution  at  the  final 

stage  N.  It  traces  from  the  value  of  x^  to  x^  recursively. 

Since  the  above  three  tables  store  only  the  feasible  and  undominated 

(for  FLIST  and  PERMUTE  tables)  solutions  at  each  stage,  the  storage  requirement 

actually  used  is  quite  different  for  different  problems.  .  Hence,  we  will  analyze 

the  worst  case  performance  of  the  algorithm.  That  is,  at  each  stage  every  point 

generated  is  feasible  and  undominated  by  the  points  at  the  previous  stage. 

Note  that  we  have  K  +  1  discretization  levels  for  each  variable  x  ,  j  *  1,  2, 

...,  N  and  M  constraints  and  one  return.  At  each  stage,  we  have  K  +  1  times  the 

number  of  points  in  the  previous  stage.  By  the  above  assumption,  no  points  are 

N 

infeasible  and  dominated.  Hence,  at  the  final  stage  N,  each  table  has  (K  +  1) 

N 

elements  and  the  storage  requirements  for  the  main  three  tables  are  2(K+I)  (M+l) 

+  (K+1)N  (M+2). 

6.4.3)  Storage  Requirement  for  the  Feasibility  and  Dominance  Test. 

As  shown  in  the  feedforward  loop  example,  the  algorithm  discards  the  infeasible 
or  dominated  points  at  each  stage.  Also,  the  remaining  feasible  and  undominated 
solutions  need  to  be  sorted  for  the  PERMUTE  tables.  The  itemized  storage  require¬ 
ments  can  be  obtained  from  the  computer  algorithm.  Here,  the  total  storage  require¬ 
ment  for  the  feasibility  and  dominance  tests  is  given  by 
6N  +  4  (M  +  1)  +  (K  +  1)? 

Now,  the  above  three  storage  requirmennts  in  the  imbedded  state  space  technique 
can  be  summarized  as  follows: 
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a)  Storage  requirement  for  the  input  data  =  (M+1)KN+M 

N  N 

b)  Storage  requirement  for  the  three  main  tables  =-2(K+l)  (M+l)  +  (K+l)  (M+2) 

c)  Storage  requirement  for  the  feasibility  and 

dominance  test  =  6N  +  4 (M+l)  +  (K+l)^ 

The  total  is  a)  +  b)  +  c)  or  (K+1)N  (3M+5)  +  KN(M+1)  +  6N  +  5M  +  4 

6.5  Discussion  on  the  General  Nonserial  D.P,  Network  Algorithm  and  the 

Imbedded  State  Space  Technique 

Let  us  compare  the  two  approaches  using  the  following  feedforward  loop 
system  with  N  stages  as  our  leitmotif. 


Figure  6.2:  A  General  N'onserial  Feedforward  Loop  Network 


The  general  nonserial  D.P.  network  algorithm  solves  the  following  problem: 
(PI)  S 


max  £  r  (x  ,  d  ) 
,  n  n  n 
n=l 


S-C-  Cn(V  dn)  =  Vl 

W  Xoj  ’  W  ’  xj-l 

Cc+i(xe+i’  d^+i)  *  xoj 


n^j ,  £+1 


x  e  X  ,  d  e  D 
n  n  n  n 

On  the  other  hand,  the  imbedded  state  space  technique  was  used  to  solve  the  following 
nonlinear  knapsack  problems: 
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(P2) 


max  z  rj(xj) 


j=l 


N 


s.t.  £  b.  ,  i  -  1,  2,  • 

j-1  J 


. .  ,  M 


“j  e  X} 


By  comparing  the  two  problems  PI  and  P2,  it  is  clear  that  if  we  delete  the  decision 


variables  d  ,  n  =  1,  2,  . ..,  N,  then  we  have  the  same  objective  function.  However, 
n 


notice  that  the  two  problems  are  quite  different  in  the  structure  of  the  constraints 
sets.  In  PI,  the  constraints  are  given  as  the  transition  function  at  each  stage 
in  the  network,  while  they  are  given  in  inequality  forms  in  P 2.  Also  notice  that 
the  RHS  of  each  constraint  of  PI  is  not  a  constant  as  in  P2,  but  still  a  variable. 
Now  suppose  that  the  constraint  in  P2  is  changed  as  N  separate  ones  at  each  stage, 
i.e. 


j  <  xj )  ib^,  for  i»l,  ...»  M,  j=l,  ...,  N. 


Then,  =  (g^ ,  g2y  •  ••,  gmj)  can  be  considered  as  a  vector  transition  function  at 


stage  j,  j  =  1,  ...,  N.  However,  at  the  junction  stage  j  in  the  Figure  6.2,  we 

have  two  separate  transition  functions  g. (x.)  and  g  (x  )  as  opposed  to  t,(x  ,  x  .) 

J  1  J  J  J  3  ®  J 

in  PI  when  the  decision  variables  are  deleted.  Hence,  if  the  transition  function  at 


the  junction  stage  j  is  separable,  i.e.. 


W  V  "  W  + 


then  the  use  of  the  imbedded  state  space  technique  may  give  a  clue  to  the  solution 
of  general  nonserial  network  problems.  However,  the  inequalities  in  the  con¬ 
straints  and  the  difference  in  the  RHS  in  the  two  problems  will  seem  to  obstruct 
the  direct  use  of  the  technique  for  the  general  nonserial  network  structures.  Never¬ 
theless,  the  ISST  has  considerable  potential  for  use  in  solving  large  scale  state 
vector  dimensional  problems  in  nonserial  networks.  Its  utility  will  be  enhanced  by 
the  us  of  fatho'  ng  approaches  such  as  those  suggested  in  the  hybrid  algorithms 
describe,  n  (2i). 
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